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evaluated  during  the  acquisition  process.  Changes  in  laws  and  procurement 
procedures  also  impacted  execution  of  the  jeep  development  program. 
Eventually,  a  single  vehicle  design  was  standardized  and  produced  during 
the  war,  primarily  by  Willys-Overland,  but  also  by  Ford.  The  design  of  the 
jeep  has  endured  as  an  acquisition  success  story.  Lessons  learned  from  the 
jeep  development  can  still  be  applied  to  systems  acquisition  programs  today. 

A  New  Look  at  Enablers  and  Barriers  to  Performance  Based  Life 
Cycle  Product  Support  (PBL)  Implementation 

p  376  Thomas  R.  Edison  and  Andre  Murphy 

Efficient  and  effective  product  support  development  and  implementation 
are  not  simple.  Increasingly,  more  focus  is  being  placed  on  how  to  deliver 
cost-wise  and  effective  product  support.  In  an  environment  of  Better  Buying 
Power— greater  efficiency  and  productivity  in  defense  spending— a  need 
to  better  understand  and  implement  product  support  that  is  performance 
outcome-based  is  not  only  prescribed,  but  prudent.  PBL  can  provide  desired 
performance  based  product  support.  A  2005  study  unearthed  perceived 
PBL  enablers  and  barriers.  This  article  is  a  byproduct  of  2011  research 
contrasting  the  2005  study’s  PBL  barriers  and  enablers.  Through  survey  of 
the  acquisition  workforce,  data  were  collected  on  15  PBL  implementation 
factors.  This  article  discusses  current  working  perceptions  that  either 
encourage  or  impede  PBL  implementation. 

Running  With  Scissors:  Defense  Budget  Cuts  and  Potential 
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Nearly  everyone  can  relate  to  the  experience  of  seeing  a  dangerous  sequence 
of  events  unfold.  A  well-intentioned  action  is  followed  by  a  subtle  misstep. 
Add  in  a  measure  of  unpredictability,  and  quickly  the  sequence  starts  to 
diverge.  In  these  situations,  a  reasonable  person  mentally  fast-forwards  to 
anticipate  the  possible  outcome.  It  is  that  quick  mind’s  eye  picture  that  spurs 
action.  It  prompts  intervention.  Building  on  the  analysis  and  recommenda¬ 
tions  presented  in  this  article,  the  author  makes  the  case  that  it  is  possible 
for  both  the  U.S.  Department  of  Defense  and  the  U.S.  defense  industry  to 
mitigate  the  dangerous  downside  risk  of  anticipated  defense  budget  cuts. 
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A  common  problem  in  defense  acquisition  is  the  difficulty  in  ensuring  that 
the  required  capabilities  stated  in  capability  development  documents  are 
technically  feasible,  affordable,  and  available  through  mature  technolo¬ 
gies.  This  problem  is  driven  by  a  lack  of  knowledge  on  both  the  capability 
developer  and  program  manager  teams.  Addressing  this  knowledge  gap 
requires  a  new  approach  to  capability  development,  where  knowledge 
gained  early  in  the  process  is  injected  into  the  capability  development 
process  in  a  rigorous  way.  This  article  describes  that  new  technical 
approach  along  with  lessons  learned  on  two  large  acquisition  programs. 
Key  tenets  include  the  use  of  pre-planned  knowledge  points  as  a  vehicle 
for  expanded  collaboration  between  program  managers  and  capability 
developers,  and  early  use  of  systems  engineering  fundamentals. 


Improving  Acquisition  Outcomes  Through  Simple  System 
Technology  Readiness  Metrics 
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This  article  advocates  the  use  of  simple  technology  readiness  metrics 
that  focus  on  system-wide  technological  maturity.  Current  DoD  practice 
is  to  set  guidelines  for  the  maturity  of  individual  system  components,  but 
the  statistical  evidence  provided  in  this  article  demonstrates  that  more 
holistic  metrics  should  be  adopted.  A  simple  system  technology  readiness 
metric  is  proposed  and  evaluated  based  on  historical  cost  and  schedule 
performance,  and  is  shown  to  be  potentially  quite  useful  in  avoiding  poor 
acquisition  outcomes.  Finally,  the  policy  implications  of  implementing  a 
decision  rule  based  on  the  metric  are  explored  in  depth,  and  the  DoD  is 
advised  to  pursue  and  encourage  applied  research  for  the  development  of 
more  comprehensive  technology  readiness  metrics. 
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From  the  Chairman 
and  Executive  Editor 


It  is  my  pleasure  to  note  several  changes  to  the 
masthead  of  the  Defense  Acquisition  Research  Jour¬ 
nal.  First,  on  May  25,  2012,  the  Senate  confirmed 
Frank  Kendall  and  Katrina  McFarland  as  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics,  and  Assistant  Secretary  of  Defense  for  Acquisition,  respec¬ 
tively.  McFarland  leaves  behind  a  distinguished  tenure  as  President  of 
the  Defense  Acquisition  University  (DAU),  a  position  now  filled  on  an 
acting  basis  by  Dr.  James  McMichael. 

The  other  change  is  the  appointment  of  Dr.  Mary  Redshaw  as  Dep¬ 
uty  Executive  Editor.  Dr.  Redshaw,  who  is  also  the  Deputy  Director  of 
Research  at  DAU,  brings  21  years  of  uniformed  military  service  and  two 
decades  of  public  and  private  experience  supporting  the  military  and 
federal  government.  Welcome  aboard! 

This  issue's  theme  “The  Military-Industrial  Complex”  is  taken  from 
the  term  popularized  by  President  Dwight  Eisenhower  in  his  farewell 
address  on  January  17, 1961.  Although  he  used  the  phrase  in  a  cautionary 
sense— “we  must  guard  against  the  acquisition  of  unwarranted  influence 
...  by  the  military-industrial  complex”— the  close  link  between  military 
and  industrial  strength  has  long  been  recognized  as  an  essential  ingredi¬ 
ent  in  a  nation’s  strategic  capability. 

The  first  article,  “The  Jeep  at  70”  by  Brian  Duddy,  reflects  the  les¬ 
sons  learned  from  the  cooperative  development  between  the  U.S.  Army 
and  several  automobile  manufacturers  to  create  the  iconic  vehicle  that 
carried  Eisenhower  and  millions  of  other  warfighters  across  every  con¬ 
tinent.  Thomas  R.  Edison  and  Andre  Murphy,  in  their  article  “A  New 


Look  at  Enablers  and  Barriers  to  Performance  Based  Life  Cycle  Product 
Support  (PBL)  Implementation  ,”  look  at  the  enablers  and  barriers  for  cre¬ 
ating  and  sustaining  effective  military-industrial  logistics  partnerships. 

The  article  “Running  with  Scissors”  by  Bryan  Riley  has  the  dis¬ 
tinction  of  winning  the  DAU  2011  Award  for  Excellence  in  Research 
and  Writing  at  the  Industrial  College  of  the  Armed  Forces  (ICAF),  now 
renamed  the  Dwight  D.  Eisenhower  School  for  National  Security  and 
Resource  Strategy  (The  Eisenhower  School)  under  the  National  Defense 
University.  Riley’s  article  is  a  cautionary  tale  of  the  risks  to  the  military- 
industrial  complex  in  the  face  of  expected  U.S.  defense  budget  cuts. 

The  last  two  articles  provide  some  rays  of  hope  in  these  otherwise 
gloomy  scenarios.  Chad  Dacus’  article  suggests  that  the  cost  and  sched¬ 
ule  risks  associated  with  immature  technologies  and  integration  can  be 
mitigated  using  a  more  holistic  means  of  measuring  system  technology 
readiness.  Mark  Pflanz  and  his  coauthors  describe  successes  and  lessons 
learned  in  applying  early  systems  engineering  concepts  to  the  develop¬ 
ment  of  the  Joint  Light  Tactical  Vehicle  program. 

In  this  issue,  John  Schank  adds  to  our  understanding  of  how  defense 
acquisition  works  (and  sometimes  doesn’t  work)  overseas,  in  his  review 
of  the  book  The  Collins  Class  Submarine  Story:  Steel,  Spies,  and  Spin. 
Finally,  we  take  the  opportunity  to  thank  all  the  reviewers  of  the  ARJ 
articles  for  the  year  2012. 


Dr.  Larrie  D.  Ferreiro 
Executive  Editor 
^  Defense  ARJ 
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Dr.  Mary  Redshaw 

Deputy  Executive  Editor 


The  Defense  Acquisition  University  (DAU)  wel¬ 
comes  the  appointment  of  Dr.  Mary  Redshaw  as 
Deputy  Executive  Editor  of  the  Defense  Acquisition 
Research  Journal  and  Deputy  Director  of  Research 
at  DAU.  She  brings  21  years  of  uniformed  military 
service  and  two  decades  of  public  and  private  experience  supporting  the 
military  and  federal  government.  Dr.  Redshaw  has  a  broad  and  varied 
background  from  which  to  draw  in  her  new  collateral  duty  as  Deputy 
Executive  Editor  for  the  Defense  Acquisition  Research  Journal  through 
such  varied  assignments  as  editor  ofth  e  Pioneers’ Progress  (1989-1991) 
and  as  a  contributing  editor  and  chapter  author  for  the  Systems  Engineer¬ 
ing  Handbook  published  in  2006  by  the  International  Council  on  Systems 
Engineering  (INCOSE). 


Her  military  career  included  assignments  as  communications 
technician,  surface  warfare  officer,  naval  flight  officer,  and  aerospace 
engineering  duty  officer.  She  became  the  first  woman  in  the  U.S.  Navy’s 
history  to  achieve  warfare  qualifications  in  both  surface  and  avia¬ 
tion  operations.  As  an  acquisition  professional  whose  career  spanned 
military  and  civilian  assignments,  Dr.  Redshaw  was  instrumental  in 
milestone  decisions  for  multiple  items  of  user  equipment  in  the  Navy’s 
Global  Positioning  System  Program  Office  and  for  the  ACAT  IC  Joint 
Standoff  Weapon. 

After  retiring  from  active  duty  in  1994,  she  supported  or  led  research 
and  acquisition  programs  for  the  Defense  Advanced  Research  Projects 
Agency,  Office  of  the  Secretary  of  Defense  (Health  Affairs),  and  the  Naval 
Air  Systems  Command  before  accepting  a  position  with  the  DAU  faculty 
in  2003.  At  DAU,  Dr.  Redshaw  developed  DAU's  online  Level  II  Systems 
Engineering  Course,  receiving  accolades  for  her  efforts  from  the  high¬ 
est  levels  of  DAU  and  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics. 


A  proponent  of  life-long  learning,  Dr.  Redshaw  earned  an  undergrad¬ 
uate  degree  in  Engineering  Science;  graduate  degrees  in  Aeronautical 
Engineering,  Business  Administration,  and  National  Resource  Strategy; 
and  advanced  degrees  in  Engineering  Management  (PhD)  and  Edu¬ 
cational  Leadership  (EdD).  She  completed  DoD’s  Senior  Acquisition 
Course  while  attending  the  Industrial  College  of  the  Armed  Forces  at 
National  Defense  University  as  well  as  the  former  20-week  Program 
Management  Course  and  the  current  Level  IV  Program  Manager’s 
Course  (PMT  401)  at  the  Defense  Systems  Management  College  (DSMC). 

Dr.  Redshaw  is  Level  III  certified  in  program  management,  sys¬ 
tems  engineering,  and  test  and  evaluation.  Additionally  she  achieved 
certification  through  two  prominent  international  associations  as  a 
Project  Management  Professional  and  a  Certified  Systems  Engineer¬ 
ing  Professional. 

Dr.  Redshaw  will  continue  to  serve  in  her  current  capacity  within  the 
Executive  Program  Management  Center,  DSMC,  a  DAU  business  unit 
co-located  with  the  university's  headquarters  at  Fort  Belvoir,  Virginia. 
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eligible  for  cash  awards  unless  the 
paper  was  researched  or  written  as 
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If  the  research  effort  is  performed 
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government  time,  the  employee  is 
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to  a  Combined  Federal  Campaign- 
registered  charity  of  winner’s  choice. 

•  First  prize  is  $1,000.  Second  prize 
is  $500. 


•  The  format  of  the  paper  must  be 
in  accordance  with  guidelines  for 
articles  submitted  for  the  Defense 
Acquisition  Research  Journal. 

•  Papers  are  to  be  submitted  to  the 
DAU  Director  of  Research: 
research@dau.mil. 

•  Papers  will  be  evaluated  by  a  panel 
selected  by  the  DAUAA  Board  of 
Directors  and  the  DAU  Director 
ofResearch. 

•  Award  winners  will  present  their 
papers  at  the  DAU  Acquisition 
Community  Training  Symposium, 
Tuesday,  April  9, 2013,  at  the  DAU  Fort 
Belvoir  Campus. 

•  Papers  must  be  submitted  by 
December  1, 2012,  and  awards  will  be 
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DAU  Center  for 
Defense  Acquisition 
Research 

Research  Agenda  2012-2013 


The  Defense  Acquisition  Research  Agenda  is  intended  to  make  researchers 
aware  of  the  topics  that  are,  or  should  be,  of  particular  concern  to  the 
broader  defense  acquisition  community  throughout  the  government, 
academic,  and  industrial  sectors.  The  purpose  of  conducting  research 
in  these  areas  is  to  provide  solid,  empirically  based  findings  to  create  a 
broad  body  of  knowledge  that  can  inform  the  development  of  policies, 
procedures,  and  processes  in  defense  acquisition,  and  to  help  shape  the 
thought  leadership  for  the  acquisition  community.  Each  issue  of  the  Defense 
ARJ  will  include  a  different  selection  of  research  topics  from  the  overall 
agenda,  which  is  at:  http://www.dau.mil/research/Pages/researchareas.aspx 

Effects  of  industrial  base 


•  What  are  the  effects  on  program  cost,  schedule,  and  performance  of 
having  more  or  fewer  competitors?  What  measures  are  there  to  deter¬ 
mine  these  effects? 

•  What  means  are  there  (or  can  be  developed)  to  measure  the  breadth 
and  depth  of  the  industrial  base  in  various  sectors,  that  goes  beyond 
simple  head-count  of  providers? 

•  Has  the  change  in  industrial  base  resulted  in  actual  change  in  output? 
How  is  that  measured? 


Competitive  contracting 

•  Commercial  industry  often  cultivates  long-term,  exclusive  (non¬ 
competitive)  supply  chain  relationships.  Does  this  model  have 
any  application  to  defense  acquisition?  Under  what  conditions/ 
circumstances? 

•  What  is  the  effect  on  program  cost,  schedule,  and  performance  of 
awards  based  on  varying  levels  of  competition:  1.  “Effective”  competi¬ 
tion  (two  or  more  offers);  2.  “Ineffective  Competition”  (only  one  offer 
received  in  response  to  competitive  solicitation);  3.  Split  awards  vs. 
winner  take  all;  and  4.  Sole  source. 


October  2012 


Comparative  studies 

•  Compare  the  industrial  policies  of  military  acquisition  in  different 
nations  and  the  policy  impacts  on  acquisition  outcomes. 

•  Compare  the  cost  and  contract  performance  of  highly  regulated  public 
utilities  with  nonregulated  “natural  monopolies,”  e.g.,  military  satel¬ 
lites,  warship  building. 

•  Compare  contracting/competition  practices  between  DoD  and  complex, 
custom-built  commercial  products  (e.g.,  offshore  oil  platforms). 

•  Compare  program  cost  performance  in  various  market  sectors:  highly 
competitive  (multiple  offerors),  limited  (two  of  three  offerors),  monopoly. 

•  Compare  the  cost  and  contract  performance  of  military  acquisition 
programs  in  nations  having  single  “purple”  acquisition  organizations 
with  those  having  Service-level  acquisition  agencies. 

Acquisition  of  services 

•  What  means  are  there  (or  can  be  developed)  to  measure  the  efficiency 
and  effectiveness  of  service  contractors? 

•  What  means  are  there  (or  can  be  developed)  to  evaluate  and  compare 
incentives  for  small  business  to  participate  in  service  contracts? 

•  What  means  are  there  (or  can  be  developed)  to  evaluate  and  compare 
services  that  the  government  outsources  with  those  kept  in-house?  Can 
they  be  used  to  evaluate  “inherently  governmental”  functions? 

•  Are  there  insights  from  other  sectors  that  can  inform  how  we  measure 
acquisition  of  services? 

•  Are  there  insights  from  the  private  sector  that  can  inform  how  we 
measure  supply  chain  management? 
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The  Jeep  at  70: 

A  Defense  Acquisition 
Success  Story 


Brian  J.  Duddy 


The  successful  development  of  the  jeep  during  World 
War  II  (WWII)  was  a  long  process  of  requirements 
development,  testing,  and  experimentation  of  small 
reconnaissance  cars,  and  incorporation  of  lessons  learned 
from  WWI  transport  vehicles.  The  jeep  prototype  was 
initially  developed  by  American  Bantam  Company,  but 
later  designs  by  Willys-Overland  and  Ford  were  also 
evaluated  during  the  acquisition  process.  Changes  in  laws 
and  procurement  procedures  also  impacted  execution 
of  the  jeep  development  program.  Eventually,  a  single 
vehicle  design  was  standardized  and  produced  during  the 
war,  primarily  by  Willys-Overland,  but  also  by  Ford.  The 
design  of  the  jeep  has  endured  as  an  acquisition  success 
story.  Lessons  learned  from  the  jeep  development  can 
still  be  applied  to  systems  acquisition  programs  today. 
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The  enormously  successful  vehicle  we  know  today  as  the  “jeep”  was 
born  out  of  requirements  developed  by  the  U.  S.  Army  prior  to  World  War 
II  (WWII).  From  a  small  company  and  simple  beginnings  came  one 
of  the  iconic  symbols  of  WWII  and  arguably  one  of  the  most  enduring 
automotive  designs  of  all  time.  The  successful  development  of  the  jeep 
demonstrates  the  need  for  requirements  harmonization,  and  the  mass 
production  and  longevity  of  the  design  demonstrates  the  application  of 
thoughtful,  long-range  manufacturing  planning.  In  this  follow-up  case 
study  to  the  P-51  Mustang  (Haggerty  &  Wood,  2010),  we  look  at  how 
the  jeep  design  was  created  and  what  set  it  on  the  course  to  become  the 
vehicle  with  the  longest  production  run  in  U.S.  history. 

The  overall  requirement  for  the  vehicle  that  became  the  jeep  followed 
the  demise  of  the  horse  as  a  method  of  military  transport  and  reconnais¬ 
sance.  Following  WWI,  the  world’s  armies  focused  their  energies  on  the 
development  of  petrol-powered  vehicles  of  all  types.  In  the  United  States, 
the  Army  was  interested  in  a  vehicle  that  could  replace  both  the  horse 
and  the  motorcycle  in  the  scout,  reconnaissance,  communication,  and 
liaison  roles  as  well  as  a  vehicle  large  enough  to  carry  the  heavy  weapons 
and  ammunition  required  by  infantry  companies. 

Army  Requirements  Develop 

The  lean  interwar  years  1919-1939  were  a  time  of  experimentation 
in  the  new  concepts  of  mechanized  warfare.  The  Army  was  searching  for 
a  solution  to  its  vehicle  requirements  during  a  time  of  dynamic  change. 
Wars  of  the  future  would  likely  be  more  mobile  than  the  trench  warfare  of 
WWI,  and  armies  would  require  a  wide  range  of  mechanized  forces.  Ini¬ 
tially,  the  Army’s  need  was  for  a  vehicle  that  would  have  a  low  silhouette, 
be  able  to  carry  a  one-  or  two-man  crew  and  a  machine  gun  with  ample 
ammunition.  It  had  to  have  speed,  toughness,  and  a  useful  payload.  The 
vehicle  also  needed  to  have  good  ground  clearance  and  cross-country 
mobility  as  it  was  envisioned  that  it  would  travel  off-road  more  often 
than  not.  The  Army  sponsored  limited  evaluations  of  a  number  of  exist¬ 
ing  light  vehicles  to  find  a  suitable  solution  to  its  requirements.  Several 
tracked  vehicles  were  tried  at  Aberdeen  Proving  Ground,  but  they  proved 
to  be  unsuitable.  The  Army  even  evaluated  a  radical  two-man  powered 
cart  developed  by  Army  personnel.  The  low-slung  vehicle  was  equipped 
with  a  machine  gun  and  nicknamed  “The  Belly  Flopper.”  Although  this 
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extensive  experimentation  and  testing  did  not  ultimately  produce  a 
vehicle  for  the  Army,  it  served  to  refine  the  requirements  for  the  light 
reconnaissance  car. 

In  1938  and  again  in  1939,  the  Pennsylvania  National  Guard  used 
a  few  small  open-topped  sedans  built  by  the  American  Bantam  Car 
Company  as  utility  vehicles  during  training  exercises.  Bantam  special¬ 
ized  in  small,  inexpensive  cars  and  they  performed  reasonably  well  in 
these  evaluations.  Based  on  the  results  of  these  exercises  and  using  the 
Bantam  design  as  a  departure  point,  the  Army  solidified  its  need  for  a 
reconnaissance  car.  It  took  many  of  the  ideas  and  design  concepts  that 
emerged  from  the  various  trials  and  demonstrations  and  merged  them 
into  a  single  set  of  general  requirements  (Denfeld  &  Fry,  1973). 

At  this  point  there  was  some  internal  disagreement  within  the  Army 
regarding  the  management  of  the  vehicle  program.  In  the  pre-WWII 
era,  acquisition  of  transport  vehicles  was  the  responsibility  of  the  Quar¬ 
termaster  Corps.  Acquisition  of  tactical  and  combat  vehicles  was  the 
responsibility  of  the  Ordnance  Corps;  the  new  vehicle  had  the  potential 
to  fill  several  different  roles,  both  tactical  and  nontactical  (Rifkind, 
1943).  Although  the  vehicle  was  initially  conceived  as  a  general  purpose 
commercial  vehicle  without  armor,  it  had  the  potential  to  evolve  into  a 
vehicle  that  could  serve  in  several  roles  in  the  combat  forces.  The  Army 
resolved  the  situation  by  appointing  an  Ordnance  Technical  Committee 
to  lead  the  program— headed  by  the  Ordnance  Corps,  but  with  represen¬ 
tatives  from  the  Infantry,  Cavalry,  and  Quartermaster  branches  (Denfeld 
&  Fry,  1973). 

The  Ordnance  Technical  Committee  was  charged  with  developing  a 
specification  for  the  vehicle  that  would  satisfy  the  needs  of  all  the  using 
branches.  This  required  difficult  compromises  on  specific  vehicle  fea¬ 
tures  and  characteristics  desired  by  each  branch.  Committee  members 
had  to  balance  such  needs  as  durability  and  cross-country  capability 
with  the  desire  for  a  vehicle  that  had  a  low  profile,  good  fuel  economy, 
and  adequate  carrying  capacity.  It  was  a  challenge  to  synthesize  all  these 
needs  into  a  vehicle  design  that  was  also  affordable  and  producible.  With 
much  effort,  eventually  a  single  specification  resulted  in  May  1940  and 
was  forwarded  to  the  Assistant  Secretary  of  War  for  approval.  The  Quar¬ 
termaster  General  directed  the  Motor  Transport  Procurement  Branch 
to  initiate  purchase  of  the  vehicle  that  would  fulfill  this  specification 
(Rifkind,  1943). 
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Detailed  Specifications  for  the  New  Vehicle 

The  details  of  the  reconnaissance  car  specification  drawn  up  by  the 
Technical  Committee  were  as  follows:  The  Vi-ton  vehicle  had  to  have 
4-wheel  drive,  a  maximum  weight  of  1,200  pounds,  a  useful  load  of  600 
pounds,  a  maximum  height  of  36  inches,  and  a  wheelbase  of  75  inches. 
The  body  style  was  to  be  rectangular  with  bucket  seats  and  a  fold-down 
windshield.  Performance  requirements  included  a  minimum  top  speed 
of  50  mph  and  a  minimum  sustained  speed  of  3  mph. 

As  soon  as  the  requirements  were  formalized,  a  small  group  of  Army 
officers  and  civilians  visited  the  Bantam  factory  in  Butler,  Pennsylva¬ 
nia,  to  further  test  Bantam  vehicles  and  discuss  the  concept  of  the  new 
military  car  with  the  Bantam  development  group  (Denfeld  &  Fry,  1973; 
Rifkind,  1943).  The  results  of  this  meeting  allowed  the  Army  to  continue 
to  refine  the  specifications  and  even  sketch  a  rough  outline  of  what  the 
new  vehicle  should  look  like.  Thus,  by  working  with  industry  the  Army 
had  arrived  at  a  set  of  requirements  that  was  simple,  functional— and 
most  importantly— achievable  (Denfeld  &  Fry,  1973). 

On  June  27, 1940,  the  Ordnance  Technical  Committee  issued  its 
final  recommendations  for  a  Vi-ton,  4x4  truck.  (The  term  4x4  meant  the 
vehicle  had  four  wheels,  all  of  which  were  powered.)  The  vehicle  maxi¬ 
mum  weight  was  now  raised  to  1,300  pounds  with  a  600-pound  payload, 
and  the  wheelbase  was  increased  to  80  inches  (Probst,  1976;  Wells,  1946; 
Vanderveen,  1971;  Denfeld  &  Fry,  1973).  To  keep  the  design  simple,  the 
Army  intended  for  manufacturers  to  use  several  common  pieces  of  mili¬ 
tary  vehicle  equipment  already  available  such  as  tail  lights  and  towing 
pintles.  The  Army  sent  invitations  to  bid  on  70  “pilot”  trucks  or  sample 
models  to  135  manufacturers.  Bidding  instructions  mandated  that  the 
first  pilot  model  should  be  delivered  to  Camp  Holabird  in  Baltimore  in  49 
days.  It  was  one  of  the  first  examples  of  “try-before-buy”  ever  used  by  the 
Army,  which  prior  to  WWII  had  been  directed  to  purchase  commercial 
off-the-shelf  trucks  almost  exclusively  (Thomson  &  Mayo,  1960). 

As  the  vehicle  development  evolved,  changes  were  also  occurring 
in  the  way  the  Army  acquired  equipment.  These  changes  were  intended 
to  help  speed  the  process  for  procuring  large  amounts  of  materiel  in  the 
event  of  a  crisis.  Many  in  the  U.S.  Government  feared  war  was  near  and 
such  process  changes  would  be  required  to  get  equipment  to  the  field  as 
fast  as  possible  versus  waiting  for  firms  to  submit  bids.  One  of  the  most 
significant  changes  occurred  in  July  1940  when  Congress  passed  Public 
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Law  703,  which  allowed  the  Services  to  negotiate  contracts  directly  with 
firms  of  their  own  choosing  rather  than  compete  programs  and  award 
contracts  to  the  lowest  bidder  (Thomson  &  Mayo,  1960). 

Out  of  135  companies  invited  to  bid  on  the  vehicle  contract,  only  two 
submitted  proposals:  American  Bantam  and  Willys-Overland  Motors. 
However,  only  Bantam  affirmed  it  could  deliver  a  vehicle  in  the  tight 
timeframe  specified  by  the  Army.  Willys  underbid  Bantam  on  per-vehicle 
cost,  but  responded  that  they  could  not  have  a  prototype  ready  for  75  days. 
On  July  25, 1940,  under  the  new  negotiated  procurement  law,  the  Army 
and  Bantam  signed  the  contract  for  delivery  of  the  pilot  vehicles  within 
the  49-day  window  (Denfeld  &  Fry,  1973). 


The  First  Jeep  is  Delivered 

Like  the  NA-73X/P-51  story,  the  prototype  machine  that  would 
eventually  mature  into  the  jeep  was  completed  in  record  time.  As  Bantam 
engineers  started  work  on  August  1, 1940,  they  knew  that  meeting  both 
the  delivery  date  and  the  specifications  would  be  extremely  difficult.  As 
the  design  solidified,  it  emerged  that  one  requirement  in  particular- 
vehicle  weight  of  only  1,300  pounds— would  not  be  achievable  in  the  short 
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term.  But  the  Bantam  team  was  experienced  enough  in  the  design  of 
small  automobiles  to  know  that  none  of  the  other  competitors  could  likely 
build  a  vehicle  that  size  and  that  robust,  which  still  fit  into  the  weight 
envelope.  Undaunted,  they  pressed  on  with  their  design.  Working  around 
the  clock,  Bantam  completed  the  prototype  and  company  executives 
drove  the  vehicle  directly  from  the  factory  in  Butler  to  Camp  Holabird, 
its  first  long-distance  trip.  On  September  23,  they  made  it  through  the 
gate  at  Holabird  with  only  30  minutes  to  spare  on  their  49-day  deadline. 

The  Bantam  vehicle,  as  delivered,  weighed  in  at  1,840  pounds  and 
was  powered  by  a  45hp  Continental  engine.  Since  the  vehicle  was  able 
to  successfully  complete  a  series  of  strenuous  tests  at  Holabird,  the 
Army  representatives  believed  that  the  maximum  weight  target  could 
be  reconsidered.  Both  the  Bantam  and  Army  engineers  knew  that  as  the 
design  matured,  strengthening  the  chassis  and  body  for  rough  service 
would  result  in  an  increase  in  the  vehicle  empty  weight.  The  addition 
of  extra  required  equipment  would  also  render  the  1,300-pound  target 
unrealistic.  In  lieu  of  a  strict  numerical  weight  objective,  the  Army  per¬ 
formance  objective  was  still  for  a  few  soldiers  to  be  able  to  manhandle 
the  vehicle  should  it  get  stuck  in  the  mud,  sand,  or  snow.  Eventually,  the 
Army  accepted  weight  growth  as  inevitable  as  long  as  that  performance 
requirement  could  be  met  (Wells,  1946).  The  focus  then  shifted  to  the 
automotive  performance  of  the  vehicle  as  more  critical  to  its  success 
than  its  weight.  In  some  ways,  the  jeep  development  employed  the  mod¬ 
ern  concepts  of  evolutionary  acquisition,  incremental  development,  and 
systems  engineering. 

Since  it  was  extremely  satisfied  with  the  prototype,  the  Army  gave 
the  go-ahead  to  Bantam  to  initiate  production  of  the  other  70  pilot 
vehicles  based  on  that  design,  but  incorporating  some  design  changes  and 
improvements  that  resulted  from  the  early  testing  of  the  prototype.  These 
changes  were  to  improve  both  performance  and  reliability.  The  testing  at 
Holabird  was  an  early  example  of  what  we  would  call  today  Test,  Analyze, 
and  Fix  or  Reliability  Development/Growth  Testing.  The  first  series  of 
the  improved  vehicles  were  delivered  to  the  Army  in  December  1940  and 
were  known  as  “Bantam  Reconnaissance  Cars”  (BRC). 

While  Bantam  was  moving  ahead  with  the  production  of  its  truck, 
Willys  was  still  in  the  competition  for  Army  contracts.  Both  Willys’ 
personnel  and  engineers  from  Ford  Motor  Company  had  been  present 
when  the  Bantam  pilot  was  delivered  to  Camp  Holabird,  so  the  company 
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had  advanced  knowledge  of  what  the  competing  vehicle  looked  like.  It 
even  had  the  opportunity  to  make  sketches  of  the  Bantam  vehicle.  The 
Army  later  responded  to  charges  that  this  represented  unfair  competi¬ 
tion  by  saying  that  the  pilot  model  was  government  property  and  that 
they  wanted  to  make  it  available  in  order  to  develop  multiple  sources 
for  production  of  the  cars  (Denfeld  &  Fry,  1973;  Jeudy  &  Tararine,  1981). 

Like  the  Bantam  team,  Willys’  engineers  also  knew  achieving  or 
beating  the  maximum  weight  requirement  specified  by  the  Army  would 
be  difficult  using  existing  technology.  Using  the  Bantam  pilot  design  as 
a  starting  point,  Willys’  designers  set  out  to  develop  a  car  roughly  fit¬ 
ting  the  other  Army  specifications,  but  incorporating  an  engine  of  their 
own  design  that  they  considered  more  suitable  for  the  mission  the  Army 
had  in  mind  for  the  vehicle  (Denfeld  &  Fry,  1973).  The  Willys  prototype 
arrived  at  Camp  Holabird  on  November  11,  1940— also  overweight  at 
2,400  pounds,  but  with  a  powerful  65hp  engine.  With  the  two  proto¬ 
types  in  place,  the  Army  decided  its  earlier  weight  goal  should  be  revised 
upward,  but  it  still  kept  a  goal  in  place  to  force  the  industry  to  consider 
weight  and  weight  savings  as  its  designs  matured. 

A  third  bidder  was  also  now  in  the  race— Ford.  The  Ford  Company 
had  decided  to  enter  the  competition  with  its  own  small  vehicle  design 
incorporating  an  existing  46hp  tractor  engine.  The  lure  of  commercial 
business  was  too  strong  for  Ford  to  stay  out  of  the  vehicle  competi¬ 
tion,  and  it  was  also  encouraged  by  the  Army  to  consider  participation 
(Denfeld  &  Fry,  1973).  All  three  companies  now  sensed  that  this  rugged 
off-road  vehicle  concept  had  the  potential  to  grow  beyond  a  military 
application.  No  vehicle  like  it  existed  in  the  civilian  world— “sport-utility 
vehicles”  were  decades  away— and  the  promise  of  extensive  commercial, 
particularly  agricultural,  sales  awaited  the  company  that  could  success¬ 
fully  secure  the  Army  contracts.  The  Ford  prototype,  called  the  “GP,”  was 
delivered  to  the  Army  on  November  23, 1940.  The  Army  now  had  three 
competing  designs  to  evaluate.  This  presented  both  a  technical  and  a 
manufacturing  challenge:  how  to  select  the  best  vehicle  design  for  the 
mission  and  ensure  that  it  could  be  produced  in  quantities  sufficient  for 
the  needs  of  a  world-wide  conflict. 

Mass  Production  Dilemma 

The  key  concern  in  getting  the  jeep  design  to  the  field  was  the  issue 
of  mass  production  capacity.  Although  Bantam  had  produced  and  refined 
the  original  design,  the  Army  believed  the  small  company  was  in  no  posi- 
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tion  to  produce  the  rate  and  number  of  vehicles  the  Quartermaster  Corps 
believed  they  would  need  for  wartime  requirements  (Thomson  &  Mayo, 
1960;  Jeudy  &  Tararine,  1986;  Zaloga,  2005).  During  WWI,  the  Army  had 
to  employ  a  dizzying  array  of  vehicles  to  meet  urgent  wartime  needs  and 
they  wanted  to  avoid  that  in  the  future;  thus,  the  goal  of  standardization 
underpinned  the  strategy  for  the  Army  vehicle  fleet  in  the  1940s  (Thom¬ 
son  &  Mayo,  1960).  Logistics  planners  did  not  want  to  repeat  the  problem 
of  provisioning  spare  parts  and  support  equipment  for  multiple  vehicle 
types  and  multiple  manufacturers. 


One  encouraging  fact  was  that  the  designs  of  all  three  prototype 
reconnaissance  cars  were  similar— a  steel  frame  and  sheet  metal  body.  It 
was  within  the  capability  of  each  of  the  three  manufacturers  to  produce 
them  since  they  were  in  some  ways  simpler  than  the  civilian  passenger 
cars  they  were  already  building.  The  central  issue  was  more  one  of  pro¬ 
duction  capacity  than  complexity,  and  there  were  a  significant  number  of 
subcontractor  components  in  each  vehicle,  particularly  the  drive  train. 

Bantam,  even  by  its  own  admission,  was  on  the  ropes.  Sales  of  its 
civilian  vehicles  were  very  modest— 1,225  in  1939  and  only  800  in  1940. 
By  the  time  of  the  jeep  design  and  competition,  it  had  no  operating  capi¬ 
tal  and  only  15  people  in  its  engineering  department  (Domer,  1976).  As 
a  result,  the  Army  looked  on  Bantam  as  high  risk  regarding  its  capacity 
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for  producing  the  thousands  of  vehicles  that  would  be  needed  for  a  global 
war.  Army  planners  initially  estimated  they  would  need  11,800  recon¬ 
naissance  cars  by  mid-1941  (Zaloga,  2005). 

Consequently,  Army  acquisition  personnel  faced  a  significant 
dilemma— do  they  stay  with  the  company  that  had  successfully  pio¬ 
neered  and  built  the  vehicle  they  wanted,  or  abandon  it  in  favor  of  a 
company  or  companies  that  could  produce  the  quantity  they  would  need 
(Thomson  &  Mayo,  I960)?  Willys  was  a  larger  company  than  Bantam,  but 
still  not  as  large  and  well-resourced  as  Ford.  The  situation  was  further 
complicated  by  the  fact  that  by  the  end  of  1940,  three  viable  yet  clearly 
different  design/prototypes  existed  from  Bantam,  Willys,  and  Ford.  If 
only  one  was  to  be  selected  for  high-rate  production  to  meet  the  goal  of 
standardization,  which  design  would  it  be?  All  three  vehicles  had  their 
own  peculiar  strengths  and  weaknesses,  and  no  single  design  was  clearly 
superior,  but  at  least  they  all  met  the  minimum  Army  requirements. 

The  Army  decided  to  solve  each  problem  in  turn.  First  to  be  settled 
was  the  design  issue,  although  the  plan  that  resulted  also  had  the  second¬ 
ary  goal  of  surfacing  potential  production  shortfalls.  To  move  the  design 
forward,  a  contract  would  go  to  all  three  manufacturers  for  500  vehicles 
each.  This  quantity  was  thought  to  be  sufficient  for  each  interested  Army 
branch  to  test  the  vehicles  thoroughly  in  an  operational  environment 
and  provide  feedback  on  the  competing  designs.  After  some  internal 
disagreements  within  the  Army,  this  plan  was  revised  to  procure  all 
1,500  vehicles  from  Bantam  on  the  grounds  that  only  its  prototype  had 
met  the  first  delivery  requirement  and  had  successfully  completed  all 
the  initial  testing,  which  the  Willys  and  Ford  models  had  yet  to  do.  This 
plan  was  revised  a  second  time  in  November  1940  to  acquiring  1,500  cars 
with  contracts  to  all  three  manufacturers,  subject  to  approval  of  each 
company’s  prototype  model.  In  consideration  of  the  new  procurement 
law  now  on  the  books,  these  were  negotiated  contracts  not  competitive 
bids  (Thomson  &  Mayo,  1960). 

All  three  manufacturers  then  set  out  to  produce  what  in  today’s 
acquisition  lexicon  would  be  “low  rate  initial  production”  quantities. 
In  the  end,  all  three  companies  had  trouble  meeting  delivery  schedules 
because  of  a  production  problem  at  the  Spicer  Company,  which  was  the 
source  for  the  axles  for  all  three  vehicle  designs.  Indeed,  the  availabil¬ 
ity  of  axles  and  related  equipment  was  a  major  production  bottleneck 
for  a  number  of  military  truck  designs  during  WWII,  since  all-wheel 
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drive  was  not  a  feature  offered  on  civilian  vehicles  and  the  only  users  of 
specialized  gear  components  and  constant  velocity  joints  were  military 
trucks.  Shortfalls  in  these  components  bedeviled  the  Army  for  many 
months  in  the  early  part  of  WWII  (Thomson  &  Mayo,  1960).  The  avail¬ 
ability  of  these  components  is  an  example  of  a  critical  technology  crucial 
to  the  supply  chain  that  drove  both  the  system  performance  and  the  total 
production  capacity  of  more  than  one  Army  truck  system. 

Operational  Testing  Results— A  Further  Dilemma 

When  these  initial  production  vehicles  reached  the  field,  they  were 
extensively  tested  by  the  Army  with  the  objective  of  selecting  the  best 
design  that  would  move  to  the  next  phase  of  high-volume  production.  The 
operational  testing  focused  mainly  on  performance  since  the  bodies  of 
all  three  vehicles  were  similar —the  Willys  and  F ord  models  having  been 
copied  from  the  original  Bantam  design.  The  weight  of  each  vehicle  had 
steadily  grown,  reinforcing  what  certain  Bantam  and  Army  engineers 
knew  was  the  case  in  practice— vehicles  tend  to  get  heavier,  not  lighter. 
That  is  still  true  today  as  all  the  prototypes  for  the  JLTV  exceeded  the 
desired  transport  weight  of  15,629  pounds  by  several  hundred  to  a  thou¬ 
sand  pounds  (Beidel,  2011).  Thus,  the  jeep  vehicle  weight  limit  increase 
overtime  is  very  similar  to  today’s  use  ofthe  threshold  and  goal/objective 
values  in  Performance  Based  Acquisition. 

In  the  performance  area,  the  Willys  “MA”  models  with  their  4-cyl- 
inder,  “Go  Devil”  engine  were  clearly  superior  (Jeudy  &  Tararine,  1986; 
Denfeld  &  Fry,  1973).  The  Willys  vehicles  also  had  the  best  acceleration 
and  cross-country  performance.  The  Bantam  models  were  notable  for 
their  superior  fuel  economy,  steering,  and  braking,  which  was  attributed 
to  Bantam’s  focus  on  keeping  the  weight  of  the  vehicle  as  low  as  possible. 
The  Ford  vehicle  came  in  third  in  the  competition,  but  it  did  have  some 
features  that  the  Army  liked  over  the  other  two  such  as  the  front-end 
design,  gear  lever,  handbrake,  and  passenger  comfort.  So  the  testing, 
while  useful,  did  not  resolve  the  dilemma  of  howto  arrive  at  a  single, 
standardized  design. 

The  Army  leadership  saw  only  two  alternatives  to  resolve  this 
dilemma:  (a)  design  a  new  vehicle  combining  all  the  desirable  features 
of  all  three  existing  designs,  or  (b)  take  the  best  design  of  the  three  exist¬ 
ing  and  graft  onto  it,  as  far  as  possible,  the  most  desirable  features  of  the 
other  two  (Cowdery,  1986;  Denfeld  &  Fry,  1973).  The  first  approach  was 
rejected  because  of  the  time  required  to  literally  “go  back  to  the  drawing 
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board”  and  design  a  new  vehicle.  At  that  time— mid -1944— the  urgency  of 
getting  vehicles  into  the  field  fast  was  becoming  the  driving  requirement. 
So  the  Army,  through  the  Quartermaster  Corps  (QMC),  would  have  to 
award  a  single  contract  for  a  “combined”  design. 

The  contract  award  decision  itself  then  became  controversial. 
Initially,  the  QMC  wanted  to  award  the  production  contract  to  Ford. 
Although  its  vehicle  came  in  last  in  the  competition,  it  was  seen  as  being 
the  lowest  risk  to  produce  the  required  number  of  vehicles  on  time.  This 
acquisition  approach  was  vetoed  by  the  government’s  Office  of  Produc¬ 
tion  Management  (OPM),  which  argued  that,  at  a  minimum,  the  contract 
should  go  to  either  Willys  or  Bantam  as  the  vehicle  designs  submitted  by 
the  two  companies  were  superior  and  both  had  met  their  earlier  contract 
requirements  (Thomson  &  Mayo,  1960;  Denfeld  &  Fry,  1973).  The  Willys 
vehicle  also  had  the  lowest  unit  price.  The  OPM  also  argued  all  along 
that  having  more  than  one  source  qualified  to  produce  the  cars  would  be 
advantageous  in  the  long  run,  particularly  if  war  loomed  on  the  horizon. 
The  positions  of  all  three  companies,  the  Roosevelt  administration,  and 
the  Army  resulted  in  a  messy  dispute,  which  even  played  out  in  the  con¬ 
temporary  press.  As  a  result  of  this  controversy,  the  Army  was  forced  to 
relent  on  the  Ford  contract  plan,  and  on  July  23, 1941,  the  QMC  awarded 
a  full-rate  production  contract  to  Willys  for  16,000  identical  jeeps  at  a 
unit  price  of  $739.00,  with  an  initial  delivery  date  of  January  1942.  This 
contract  award  reflects  what  we  would  consider  today  as  “Best  Value” 
for  the  government. 


A  Legend  is  Born 

The  task  that  remained  was  to  synthesize  the  advantageous  charac¬ 
teristics  ofthe  three  competingvehicles  into  a  single  vehicle  configuration 
that  the  Willys  team  was  to  use  as  the  production  design.  After  the  con¬ 
tract  award  to  Willys,  the  Army  convened  its  own  team  to  finalize  the 
features  of  the  vehicle  using  the  Willys  MA  as  the  new  baseline  con¬ 
figuration.  The  Army  engineers  also  had  to  ensure  inclusion  of  other 
standardized  military  vehicle  equipment  into  the  final  configuration. 
The  design  that  resulted,  the  Willys  MB  “y4-ton,  4x4  utility,”  was  broadly 
the  Willys  body,  chassis,  engine,  and  drive  train  with  a  Ford  front  end 
and  grille.  This  configuration  would  become  the  standard  for  more  than 
640,000  WWII  jeeps— an  iconic  design  that  would  last  70  plus  years  and 
spawn  an  entire  new  class  of  civilian  vehicle.  The  jeeps  that  were  provided 
to  U.S.  and  Allied  forces  during  the  war  were  an  instant  success.  A  vehicle 
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originally  designed  for  use  by  the  Combat  Arms  branch  was  adopted  and 
used  by  all  the  Services.  In  fact,  the  vehicle  was  so  versatile  in  a  num¬ 
ber  of  roles  and  in  all  theaters  of  the  war  that  its  reputation  grew  well 
beyond  anything  the  original  designers  could  have  imagined.  Although 
its  programmatic  beginnings  were  rocky,  the  little  car  quickly  became  an 
overnight  operational  success  story  and  an  instant  hit  with  G.I.’s. 

What  became  of  the  original  three  companies?  As  forecast  by  the 
OPM,  as  soon  as  the  United  States  became  involved  in  WWII,  a  sec¬ 
ond  source  was  needed  to  produce  the  jeep  in  addition  to  those  being 
produced  by  Willys,  which  eventually  produced  362,000  MB  models. 
That  second-source  contract  went  to  Ford,  which  by  the  end  of  1945, 
co-produced  277,000  of  its  own  version  of  the  Willys  MB .  The  Ford  ver¬ 
sion,  the  GPW,  differed  in  only  the  smallest  details  from  the  Willys  and 
allowed  the  Army  to  achieve  the  standardization  and  production  volume 
it  desired.  Bantam,  however,  lost  out  completely  on  subsequent  Army  jeep 
contracts,  producing  a  total  of  only  2,600  vehicles,  many  of  which  went 
to  Allied  nations.  Congressional  hearings  were  eventually  held  on  the 
controversy  of  who  had  “invented”  the  jeep,  and  Bantam  was  vindicated 
by  the  judgment  of  the  U.S.  Government  that  the  wartime  design  was 
based  on  its  initial  prototype  and  intellectual  property  (Rifkind,  1943). 
This  victory  did  not  help  the  company  financially.  Although  Bantam  did 
produce  trailers  for  the  jeep  during  the  war,  the  company  did  not  survive 
much  past  1945.  In  truth,  the  jeep  design  was  a  product  of  a  massive  team 
effort,  including  all  three  manufacturers  as  well  as  Army  engineers,  both 
military  and  civilian  (Vanderveen,  1971;  Wells,  1946;  Hogan,  1941). 

Newer  models  of  the  Willys  jeeps  “soldiered  on”  in  the  U.S.  military 
until  the  Vietnam  War  when  they  were  gradually  replaced  with  a  new 
vehicle  design— the  Ford  M151  MUTT.  Production  of  civilian  jeeps 
began  immediately  after  WWII.  Although  based  on  newer  technology, 
the  M151  owed  much  of  its  design  to  its  predecessors,  the  Bantam  BRC 
and  the  Willys  MB.  The  jeep  was  a  design  for  the  ages  and  one  that  will 
seemingly  never  go  out  of  style.  Today’s  contemporary  civilian  model,  the 
Jeep  Wrangler,  still  maintains  many  of  the  design  features  of  the  y4-ton 
4x4  reconnaissance  car  of  WWII.  Its  design  longevity  has  made  it  a  true 
defense  acquisition  success  story. 
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Lessons  Learned 

The  lessons  learned  from  the  development  and  production  of  the 
jeep  are  many.  Surely  the  first  and  most  significant  is  the  importance 
of  government  and  industry  partnerships  to  work  together  to  satisfy 
operational  requirements.  As  the  Army,  Bantam,  and  Willys  evaluated 
the  maturity  of  the  reconnaissance  car  design,  they  grew  to  understand 
the  “art  of  the  possible.”  The  Army  involvement  in  the  jeep  development 
functioned  in  ways  similar  to  the  Department  of  Defense  (DoD)’s  current 
Integrated  Product  and  Process  Development/Integrated  Product  Team 
environment  and  the  Joint  Capabilities  Integration  and  Development 
System  process.  Developing  requirements  that  are  simple,  functional, 
and  achievable  is  a  good  model  for  the  JLT  V  program. 

A  second  key  lesson  is  impact  of  sound  production  planning  on  the 
eventual  deployment  and  sustainment  of  the  system.  The  Army  knew 
that  multiple  sources  would  be  required  to  supply  the  jeeps  for  wartime 
needs.  Although  the  methods  it  used  to  ensure  multiple  sources  were 
criticized,  in  the  end  its  strategy  was  validated  as  the  production  from 
both  Willys  and  Ford  provided  sufficient  quantities  of  a  standardized 
vehicle  for  not  only  the  United  States,  but  many  Allied  nations.  The 
operational  testing  at  Camp  Holabird  and  other  locations  surfaced  reli¬ 
ability  problems  on  the  early  vehicles,  which  would  later  be  corrected  in 
full-rate  production  versions. 

A  third  enduring  lesson  is  keeping  designs  realistic  and  having  the 
courage  to  prioritize  or  revise  requirements  in  light  of  common  sense  and 
the  results  of  operational  testing.  The  weight  limit  on  the  jeep  imposed  by 
the  Army  was  revised  based  on  technological  and  operational  realities, 
but  was  maintained  to  avoid  too  many  “ornaments”  being  added  to  the 
design.  The  development  and  production  of  the  jeep  followed  an  approach 
similar  to  DoD’s  current  Performance  Based  Acquisition  process. 

Dilemmas  in  defense  acquisition  will  always  arise,  but  by  taking 
some  lessons  from  history,  those  dilemmas  can  be  successfully  resolved. 
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A  New  Look  at  Enablers  and 
Barriers  to  Performance 
Based  Life  Cycle 
Product  Support  (PBL) 
Implementation 


Thomas  R.  Edison  and  Andre  Murphy 


Efficient  and  effective  product  support  development  and 
implementation  are  not  simple.  Increasingly,  more  focus 
is  being  placed  on  how  to  deliver  cost-wise  and  effective 
product  support.  In  an  environment  of  Better  Buying 
Power— greater  efficiency  and  productivity  in  defense 
spending— a  need  to  better  understand  and  implement 
product  support  that  is  performance  outcome-based 
is  not  only  prescribed,  but  prudent.  PBL  can  provide 
desired  performance  based  product  support.  A  2005 
study  unearthed  perceived  PBL  enablers  and  barriers. 
This  article  is  a  byproduct  of  2011  research  contrasting 
the  2005  study’s  PBL  barriers  and  enablers.  Through 
survey  of  the  acquisition  workforce,  data  were  collected 
on  15  PBL  implementation  factors.  This  article  discusses 
current  working  perceptions  that  either  encourage  or 
impede  PBL  implementation. 
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The  acronym  “PBL”  has  changed.  Its  definition  now  corresponds  to 
Performance  Based  Life  Cycle  Product  Support  vice  Performance  Based 
Logistics— but  has  the  scope  and  function  of  PBL  really  changed?  Indeed, 
the  objective  of  the  PBL  name  change  was  to  broaden  the  context  of  how, 
why,  when,  and  who  would  implement  and  manage  PBL.  Rightfully  so, 
policy  makers  thought  it  prudent  to  redefine  the  initiative  to  make  it 
clear  that  PBL  is  not  just  a  tool  for  the  logistician,  but  now  includes  other 
program  areas  of  responsibility  such  as  system  engineers,  contract  spe¬ 
cialists,  etc. 

The  work  of  various  roles  required  with  the  implementation  of  PBL  is 
broader  than  what  was  initially  envisioned,  and  PBL  has  become  a  more 
significant  enabler  to  greater  product  support  capability  throughout  the 
Department  of  Defense  (DoD).  PBL  should  no  longer  be  viewed  as  solely 
outcome-focused  on  an  end  product,  nor  from  just  the  perspective  of 
supporting  the  logistics  support  elements.  PBL  needs  to  be  considered 
throughout  the  entire  life  cycle  and  as  an  enabler  to  forge  a  more  effective 
product  support  strategy  throughout  the  product’s  entire  life  cycle— from 
“must  have  it”  (initial  requirement)  to  “rust  has  it”  (final  disposal). 

But  changing  PBL  and  how  it  is  implemented  and  managed  obvi¬ 
ously  takes  more  than  a  name  change  from  the  stroke  of  a  pen.  Has  PBL 
really  changed  over  the  years  since  it  was  embraced  in  earnest  in  the  late 
1990s?  Specifically,  has  the  PBL  environment  changed— have  barriers 
and/or  enablers  been  transformed  for  PBL  so  it  can  be  implemented  more 
successfully  throughout  DoD?  At  the  day’s  end,  do  we  really  understand 
PBL?  Have  perceptions  of  PBL  being  too  expensive,  requiring  greater 
funding,  or  being  too  complicated  to  implement  in  terms  of  develop¬ 
ing  proper  contractual  incentives/awards  or  partnering  agreements, 
changed?  Have  Services’  viewpoints  of  PBL  changed?  Have  some  of  the 
barriers  and  enablers  to  PBL’s  effective  and  efficient  implementation 
changed  over  the  last  5-10  years? 

The  research  analyzed  the  current  perceptions  of  PBL  through 
the  eyes  of  approximately  300  plus  military,  civilian  government,  and 
contractor  personnel  working  primarily  in  program  management  and 
logistics.  The  respondents  were  asked  to  rate  15  factors  as  to  whether 
they  believed  a  factor  was  a  barrier  or  an  enabler  to  PBL  implementation. 
They  also  rated  how  significant  they  believed  each  factor  impacted  PBL 
implementation  in  their  program  on  a  scale  from  1  (minimal)  to  4  (very 
significant).  They  were  asked  other  related  questions  to  determine  if  they 
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had  accomplished  a  Business  Case  Analysis  on  their  program  and  what 
the  overall  effect  was  on  their  program’s  cost,  schedule,  and  performance 
from  implementing  PBL. 

Over  600  defense  acquisition  professionals  had  an  opportunity  to 
participate  in  the  online  posted  survey.  The  invited  respondent  pool 
consisted  of  selected  graduates  of  Defense  Acquisition  University  (DAU) 
acquisition  courses  (e.g.,  LOG  235/236  and  LOG  350)  and  other  identi¬ 
fied  personnel  that  were  known  to  have  knowledge  and  experience  in 
implementing  PBL  within  DoD.  Of  the  identified  personnel  that  were 
invited  to  participate  in  the  survey,  approximately  50  percent  partici¬ 
pated  in  the  survey.  These  writings  will  explore  and  discuss  information 
gathered  from  those  300  plus  PBL  implementers  on  whether  the  effects 
of  PBL  barriers  and  enablers  have  changed.  But  before  we  discuss  more 
on  PBL  perceptions,  we  should  first  understand  where  we  have  been  to 
know  where  we  need  to  go.  A  synthesis  of  recent  writings  is  provided  to 
underscore  the  discussion  of  perceived  changes  in  PBL  implementation 
enablers  and  barriers. 
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PBL  Yesterday  and  Today 

Much  has  been  written  (Canaday,  2010;  DeVries,  2005;  Fowler,  2009; 
Fowler,  2010;  Geary,  Koster,  Randall,  &  Haynie,  2010;  Kobren,  2009; 
Miller,  2008;  Omings,  2010),  spoken,  and  taught  regarding  PBL— not  only 
about  its  advantages,  but  also  about  what  prevents  it  from  being  fully 
embraced  and  effectively  implemented  by  all  the  Services. 

Since  PBL  is  becoming  a  growing  practice  within  industry  and  DoD, 
the  literature  discussed  herein  will  leverage  both  bodies  of  knowledge 
within  industry  and  government.  Before  the  discussion  begins,  we  should 
level  the  playing  field  with  a  common  understanding  and  concise  defini¬ 
tion  of  PBL.  As  Kobren  (2009)  asserts,  PBL  is  about  performance.  It  is 
about  readiness.  It  is  also  about  enabling  mission  accomplishment  and 
ensuring  the  warfighter  has  weapon  systems  that  are  available,  reliable, 
and  supportable  when  and  where  required.  PBL  is  part  of  a  long  tradition 
of  contracting  for  performance.  Since  its  inception,  PBL  has  continued  to 
evolve.  The  shift  toward  Integrated  Logistics  Support  attempted  to  wrap 
together  the  distinct  logistics  elements  into  a  coordinated  approach, 
but  there  was  still  the  disjointed  acquisition  versus  sustainment  sup¬ 
port  issues  and  the  lack  of  a  linkage  between  supportability  measures 
and  warfighter  needs  (DeVries,  2005).  Fowler  (2009),  then  Assistant 
Deputy  Under  Secretary  of  Defense  for  Materiel  Readiness,  believes  the 
time  is  coming  to  rebrand  the  sustainment  approach.  The  rebranding 
effort  should  include  an  emphasis  on  re-integrating  complete  life-cycle 
sustainment  into  programs. 

Clearly,  product  support,  while  primarily  a  logistics  and  sustainment 
function,  is  not  actually  synonymous  with  the  fundamental  aspect  of 
logistics.  To  that  point,  product  support  encompasses  materiel  manage¬ 
ment,  distribution,  technical  data  management,  maintenance,  training, 
cataloging,  configuration  management,  engineering  support,  repair  parts 
management,  failure  reporting  and  analysis,  and  reliability  growth  (DoD, 
2009).  To  further  this  point,  Canady  (2010)  talks  about  how  PBL  remains 
the  preferred  method  for  weapon  systems  sustainment.  However,  defense 
officials  are  scrutinizing  PBL  strategies  such  as  those  on  the  C-17,  press¬ 
ing  for  lower  costs,  better  proof  of  savings,  and  more  government  control 
of  long-term  sustainment  options. 
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Geary  et  al.  (2010)  inform  us  that  effective  product  support  requires 
contributions  from  both  the  public  and  private  sectors.  A  significant 
challenge  over  the  course  of  the  next  decade,  particularly  in  today’s 
acquisition  environment  of  declining  financial  resources  combined 
with  project  deficits  and  undiminished  operational  demands,  is  creat¬ 
ing  a  more  effective,  unified,  and  fiscally  prudent  industrial  integration 
strategy  for  product  support.  They  also  highlight  some  of  the  real  DoD 
innovators  and  enablers  in  deploying  PBL  effectively  and  why  they  were 
successful.  Some  of  the  highlighted  key  enablers  to  PBL’s  success  were: 
integrated  partnerships,  incentive  strategies,  a  culture  of  innovative 
teams,  shared  visions  on  objectives/metrics/incentives,  and  shared  com¬ 
mon  grounds  on  win-win  scenarios  between  industry  and  government. 

In  government,  PBL  has  garnered  mixed  reviews  and  outcomes. 
A  few  organizations  have  implemented  support  strategies  under  the 
guise  of  a  performance  outcome-based  strategy  only  to  discover  the 
product  support  was  a  hybrid  version  of  a  transactional  arrangement. 
Department  of  Defense  Directive  5000.01  (2007)  requires  that  program 
managers  develop  and  implement  performance  outcome-based  logistics 
strategies  that  optimize  total  system  availability  while  minimizing  cost 
and  the  logistics  footprint.  But,  more  than  we  would  like  to  think,  organi¬ 
zations  proceed  at  their  own  peril  by  not  conducting  an  initial  business 
case  analysis  to  determine  their  potential  Return  On  Investment  asso¬ 
ciated  with  their  product  support  decision.  Fortunately,  there  are  true 
successful  ventures  that  evidence  those  attributes  and  objectives  sought 
with  PBL  implementation  (Beggs,  Seymour,  &  Ertel,  2005). 

Miller  (2008)  identifies  an  ingredient  required  for  a  successful  PBL 
undertaking.  Stated  plainly:  Get  on  with  the  work  of  sourcing  the  best 
possible  product  support  results  for  the  warfighter  given  statutes  and 
regulations  governing  your  options.  Find  the  most  cost-effective  means 
of  supporting  warfighters.  He  further  states,  the  research  is  clear  that, 
properly  done,  PBL  can  be  an  important  part  of  the  solution.  He  also 
highlights  several  barriers  and  enablers  that  affect  PBL  implementa¬ 
tion-similar  to  those  explored  in  this  article.  He  identifies  funding, 
regulations,  BCAs,  and  several  other  misperceptions  driven  by  a  misun¬ 
derstanding  or  lack  of  experience  working  with  PBL. 


Defense ARJ,  October  2012,  Vol.  19 No.  4 : 376-393 


381 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


Former  Deputy  Under  Secretary  of  Defense  for  Materiel  Readiness 
Randy  Fowler  wrote  in  his  2010  work,  Future  of  Product  Support,  that 
among  critics  there  remains  a  strong  consensus  that  an  outcome-based, 
performance-oriented  product  support  strategy  is  a  worthy  objective. 
As  much  as  any  other  organizational  construct  to  date,  Fowler  touches 
on  the  situation  of  defense  leadership.  On  the  one  hand,  transforming 
product  support  will  require  not  only  strong  leadership  in  DoD,  but  also 
an  open-minded,  reform-driven  DoD-congressional  partnership  and  a 
collaborative  DoD-industry  relationship  to  realize  PBL’s  objectives.  The 
national  security  and  economic  environments  dictate  tough-minded 
acquisition  reform  and  logistics  transformation.  On  the  other  hand,  the 
challenges  of  affordability  constraints;  the  need  to  upgrade  systems, 
processes,  and  infrastructure;  and  a  continuing,  persistent  operations 
tempo  prescribe  a  clear  need  for  DoD  implementation  of  an  integrated 
plan  to  address  product  support  across  the  defense  enterprise— like  PBL. 

Fowler  (2010)  also  suggests  that  PBL  will  only  succeed  when  driven 
from  the  topmost  levels  in  the  program  or  organization.  One  can  surmise 
only  top-level  managers  have  the  breadth  of  perspective  and  authority 
needed  to  see  the  entire  process  from  start  to  finish.  An  effective  pro- 
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ponent  of  PBL  must  be  part  visionary,  part  communicator,  and  part  leg 
breaker.  Program  managers  are  charged  with  ensuring  the  development 
and  implementation  of  performance  outcome-based  strategies  that 
strive  for  a  more  cost-effective  weapon  systems  support  approach  and 
a  balanced  use  of  public-private  partnerships.  Program  managers  and 
logisticians  must  be  open  to  contrasting  product  support  strategies  to 
experience  those  benefits  PBL  can  afford  a  weapon  system. 

Omings  (2010)  offers,  in  certain  circles,  that  PBL  has  been  viewed 
as  a  business  fad  and  is  derided  in  much  the  same  fashion  as  Total  Qual¬ 
ity  Management  and  Lean  Six  Sigma  when  those  concepts  were  first 
espoused— misconceptions  on  their  true  value.  He  highlights  that  it  is 
true  that  these  methods  are  not  a  panacea,  but  time  has  shown  that  when 
applied  under  the  right  circumstances,  they  can  provide  powerful  results. 

One  final  point  about  Fowler’s  discussion  on  the  future  of  product 
support  should  be  noted:  Fowler,  like  Kobren  and  Geary  et  al.,  under¬ 
stands  the  role  of  a  product  support  strategy  such  as  PBL  where  it  is 
crucial  to  our  national  interest  to  ensure  that  product  support  achieves 
a  level  of  performance  equal  to  its  importance.  Customer  or  warfighter 
requirements,  not  internal  values,  should  guide  the  product  support 
manager’s  performance  or  decisions.  They  must  replace  old  ways  of 
thinking  with  new  ideals  and  expectations  associated  with  letting  the 
old  paradigms  go.  These  include  replacing  perfectionist  ways  of  think¬ 
ing  with  experimental  thinking,  and  getting-it -just-right  credos  with 
making-it-better  credos. 

A  recurring  theme  among  authors  is  the  importance  of  positive 
preconditions  for  PBL  success:  senior  management  and  sponsorship, 
realistic  requirements  and  expectations,  empowered  and  collaborative 
product  support  integrators,  strategic  context  for  efficiency  growth, 
shared  vision,  sound  supply  chain  management  practices,  and  appro¬ 
priate  people  participating  full-time  with  a  sufficient  budget.  Some  also 
identify  negative  preconditions  related  to  PBL:  wrong  sponsor  (leader 
for  the  job),  cost-cutting  focus,  narrow  technical  focus,  and  do-it-to-me 
attitudes.  Some  authors  assert  that,  to  turn  around  negative  conditions, 
we  must  educate  the  workforce  on  PBL  and  do  something  small  first. 
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PBL  Perceptions  Discovered 

What  do  we  truly  understand  about  the  current  workforce’s 
perceptions  of  PBL?  What  are  some  of  the  known  or  perceived  PBL 
implementation  and  management  conundrums  facing  program  manage¬ 
ment  practitioners? 

The  objective  of  the  study  was  to  gather  information  from  senior  DoD 
leadership  on  factors  that  could  be  enhanced  to  help  reduce  identified 
barriers  or  factors  that  would  enable  more  effective  PBL  implementation. 
The  study  examined  the  perceived  effects  of  15  factors  relating  to  PBL 
implementation— whether  they  were  a  barrier  or  enabler  and  the  relative 
importance  of  these  factors  for  carrying  out  the  product  support  strategy. 
The  genesis  of  factors  used  in  this  and  the  previous  study  was  based  on 
literature  searches,  numerous  PBL  briefings  from  the  Services,  various 
conference  minutes,  and  informed  identification  of  what  is  perceived  to 
be  an  appropriate  set  of  factors  for  study— the  most  prevalent  barriers 
and  enablers  that  were  impacting  PBL  implementation  efforts. 

Figure  1  shows  the  factors  and  definitions  that  were  rated  on  the 
survey  by  each  of  the  300  plus  respondents.  Respondents  were  provided 
the  option  to  rate  a  neutrally  or  unbiased  (no  predisposition  to  being  an 
enabler  or  barrier)  worded  factor  as  either  a  barrier  or  an  enabler. 

The  method  used  to  determine  a  factor’s  specific  rating  score  and 
whether  a  factor  was  a  barrier  (negative  rating  score)  or  an  enabler  (posi¬ 
tive  rating  score)  was  to  multiply  the  ranking  (either  positive  or  negative 
1, 2, 3,  or  4  based  on  the  respondent’s  selection  on  the  survey)  by  the  total 
number  (votes)  of  respondents  that  selected  that  ranking. 

Information  was  obtained  suggestive  of  current  thinking  regarding 
these  15  factors  as  to  their  effects  on  PBL  implementation— the  main 
objective  of  the  survey.  Ten  factors  were  determined  by  respondents 
to  be  enablers  (positive  rating  scores)  to  implementation  while  5  were 
determined  to  be  barriers  (negative  rating  scores).  In  particular.  War¬ 
fighters’  Perspective  had  the  highest  rated  score  as  an  enabler  with  a 
score  of  323.  Five  factors  (Performance  Metrics,  Total  Life  Cycle  Sys¬ 
tems  Management  [TLCSM],  Strategic  Alliances/Partnerships,  Supply 
Chain  Management  [SCM],  and  Performance  Based  [PB]  Contracting) 
were  next  in  the  positive  rankings  (enablers)  with  similar  scores  ranging 
from  217  (Metrics  and  TLCSM)  to  193  for  PB  Contracting.  Four  other 
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FIGURE  1.  FACTORS,  DEFINITIONS,  AND  SURVEY  RESPONDENTS’ 
PERCEPTIONS  (BARRIERS  VS.  ENABLERS) 


1. 

BARRIER:  Funding.  Working  capital  fund,  colors  of  $,  expiring  $ 

2. 

BARRIER:  Statutory-Regulatory  Requirements.  Title  10,  Core, 

DoDI  5000.02,  Service  policies 

3. 

BARRIER:  Cultural  Paradigms.  Organic  vs.  Contractor  Logistics 
Support  (CLS),  parts  management  vs.  performance  management 

4. 

BARRIER:  Existing  Infrastructure  or  Organization.  Management, 
oversight/review,  structures/processes 

5. 

BARRIER:  Technical  Data  (TD)  Rights.  Ownership  of  technical 
data  package,  access  to  technical  data 

6. 

ENABLER:  PBL  Awareness/Training.  Formal  DAU  training, 
in-house/on-the-job  training,  personnel  skills 

7. 

ENABLER:  Incentives/Awards.  Award/incentive  fees, 
administration  of  innovative  contracts/agreements 

8. 

ENABLER:  Supply  Chain  Management  (SCM).  End-to-end 
customer  support,  enterprise  integration 

9. 

ENABLER:  Strategic  Alliances/Partnerships.  Depot  partnering, 
joint  ventures 

10. 

ENABLER:  Performance  Based  (PB)  Contracting.  Incentive/award 
fees,  innovative  contracts 

11. 

ENABLER:  Performance  Metrics.  Information  systems,  variations, 

trends 

12. 

ENABLER:  Total  Life  Cycle  Support  Management  (TLCSM). 

PM’s  TLC  product  support  responsibility 

13. 

ENABLER:  Adoption  of  Commercial  Off  the  Shelf  (COTS). 

Commercial  practices/procedures,  products,  subsystems 

14. 

ENABLER:  Total  Ownership  Cost  (TOC).  Cost  accounting, 
reporting,  tracking 

15. 

ENABLER:  Warfighters’  Perspectives.  Readiness,  affordability, 
combat  requirements 
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enablers  were  close  in  their  positive  rating  scores  (124  to  103):  Incen¬ 
tives/Awards,  PBL  Awareness/Training,  Adoption  of  COTS,  and  Total 
Ownership  Cost  (TOC). 

Cultural  Paradigms  was  the  highest  rated  barrier  (negative  ranking 
score  of  -300).  This  was  significantly  above  the  second  place  barrier  of 
Funding  with  -170.  The  next  three  negative  rated  factors  (barriers)  were 
grouped  together  from  -139  to  -100.  Collectively,  raw  scores  for  the  four 
grouped  barrier  factors— Funding,  Technical  Data  [TD]  Rights,  Existing 
Infrastructure  or  Organization,  and  Statutory-Regulatory  Requirements 
—were  so  similar  that  little  can  be  interpreted  about  their  relative  differ¬ 
ent  effects  on  PBL  without  further  or  a  more  granular  analysis. 

The  graphic  (Figure  2)  displays  the  rating  scores  and  relative  differ¬ 
ences  or  similarity  of  the  barriers  and  enablers. 


FIGURE  2.  ENABLERS/BARRIERS  RELATIVE  COMPARISONS 


400 


300 


200 


0 


-200 


-300 


-400 


ENABLERS 


386 


Defense ARJ,  October  2012,  Vol.  19 No.  4: 376-393 


A  New  Look  at  Enablers  and  Barriers  to  Performance  Based  Life  Cycle  Product  Support  (PBL)  Implementation 


The  survey  data  analyses  determined  that  from  the  initial  15  fac¬ 
tors,  10  were  enablers  and  5  were  barriers  to  PBL  implementation.  This 
distribution  of  factors  was  considered  significant  since  in  the  previous 
study  in  2005  by  DeVries  7  factors  were  considered  to  be  enablers  while 
7  others  were  considered  to  be  barriers.  An  additional  factor  of  Warf¬ 
ighters’  Perspective  was  included  in  this  study  that  was  not  rated  in 
2005.  Specifically,  PBL  Training  and  Incentives/Awards  had  previously 
(2005)  been  identified  as  a  barrier.  The  2011  study  determined  they  were 
enablers.  Even  though  these  two  factors  (Training  and  Incentives)  were 
rated  in  the  lower  30  percent  of  the  enablers  in  2011,  it  is  suggestive  that 
respondents  did  not  perceive  these  two  ranked  factors  to  be  barriers  as 
they  were  categorized  in  the  2005  survey.  This  highlighted  the  reason  for 
allowing  the  respondents  to  determine  by  their  ratings  whether  a  factor 
was  a  barrier  or  an  enabler. 

The  results  highlighted  that  of  the  15  factors  rated  on  the  ques¬ 
tionnaire,  only  5  were  identified  as  barriers.  Ten  were  identified  as 
enablers— 3  more  than  in  the  2005  study.  More  factors  are  now  (2011) 
considered  to  be  enablers  to  PBL  implementation  than  previously  iden¬ 
tified  in  2005.  Perceptions  in  2011  may  be  that  PBL  is  not  as  difficult  to 
implement  and  more  factors  are  considered  to  be  aids  or  enablers  to  its 
successful  implementation. 

As  highlighted  in  figure  1,  respondents  considered  the  Warfighter’s 
Perspectives  to  be  the  most  important  positive  factor  (enabler)  facilitat¬ 
ing  PBL  implementation.  This  matches  the  commonly  accepted  theory 
that  the  warfighter  is  normally  assumed  to  be  one  of  the  most  critical 
elements  or  factors  to  a  program’s  overall  success.  The  success  of  PBL 
implementation  is  no  different— the  Warfighters’  Perspectives  factor  is 
highlighted  in  this  research  data  as  being  critical  to  a  program’s  success. 

Respondents  considered  Cultural  Paradigms  a  significant  challenge 
(barrier)  to  PBL  implementation.  Cultural  paradigms  are  normally 
assumed  to  be  among  the  most  serious  impediments  or  hindering  fac¬ 
tors  to  a  program’s  ability  to  accept  change  or  accomplish  a  challenging 
issue  within  the  program.  The  success  of  PBL  implementation  is  no 
different— Cultural  Paradigms  must  be  overcome  if  a  program  or  PBL 
is  to  succeed  in  the  complex  DoD  environment.  In  relationship  to  PBL 
implementation,  Cultural  Paradigms  being  rated  the  highest  is  not  sur¬ 
prising  given  that  culture  is  the  most  challenging  factor  to  overcome  in 
any  significant  change,  especially  when  these  new  concepts  or  changes 
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are  viewed  as  threats.  Many  government  personnel  consider  PBL  as 
a  threat  because  of  a  common  misperception  that  it  is  a  synonym  for 
contractor  logistics  support  (CLS)  or  “contracting  out  support.”  The 
aforementioned  information  is  vital  for  any  program  to  consider  when 
attempting  implementation  of  different  business  practices  like  perfor¬ 
mance  based  incentives. 

As  seen  through  the  eyes  or  viewpoint  of  the  respondents,  clearly  the 
Warfighters’  Perspectives  factor  had  a  positive  effect  on  PBL  implementa¬ 
tion.  Respondents  report  that  if  they  were  able  to  determine  and  maintain 
a  warfighter’s  point  of  view,  they  had  a  greater  ability  to  effectively  imple¬ 
ment  PBL.  Clearly,  these  respondents  are  sending  an  important  message 
to  potential  implementers  of  PBL— if  you  want  to  effectively  implement 
PBL  you  need  to  understand  and  maintain  the  Warfighters’  Perspectives. 
This  point  of  view  is  a  normal,  commonly  accepted  theory,  but  one  that 
is  not  always  supported  with  empirical  data. 

The  study  also  revealed  that  the  same  can  be  said  for  Cultural  Para¬ 
digms;  it  had  aperceived  significant  effect  on  PBL— but  as  abarrier.  Like 
the  Warfighters’  Perspectives  factor,  the  Cultural  Paradigms  factor 
has  a  significant  effect  on  PBL  implementation.  The  Cultural  Para¬ 
digms  factor  is  perceived  as  a  significant  barrier  and  must  be  reduced 
or  eliminated  if  PBL  is  going  to  be  more  successful.  Specific  paradigms 
were  not  detailed  within  the  survey;  however,  some  commonly  known 
paradigms  consider  PBL  as  another  way  of  buying  CLS,  as  too  expensive 
to  incorporate  and  manage,  and  not  as  flexible  in  terms  of  providing 
needed  product  support.  PBL  can  be  a  valued-added  game  changer  such 
as  public-private  partnerships,  where  potentially  the  best  of  entities 
(industry  and  government)  collaborate  to  meet  warfighter  requirements. 
To  move  beyond  cultural  impediments  requires  hard  work  to  change 
old  ways  of  thinking.  As  discussed  earlier  in  this  article,  customer  or 
warfighter  requirements,  not  internal  values,  should  guide  the  product 
support  manager’s  performance  or  decisions.  They  must  replace  old  ways 
of  thinking  with  new  theories  and  expectations  associated  with  letting 
the  old  paradigms  go.  If  DoD  is  to  effectively  implement  PBL,  then  the 
acquisition  and  sustainment  workforce  education  and  training  needs 
to  continue  the  reduction  of  cultural  maladies  that  impede  the  ability 
to  implement  a  viable  PBL  solution. 
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Results  and  Discussion 

The  Warfighters’  Perspectives  are  perceived  by  senior  program 
management  practitioners  to  be  the  most  vital  enabling  factor  to  ensur¬ 
ing  PBL  is  effectively  and  efficiently  implemented— a  result  suggestive  of 
the  2011  study.  In  all  future  endeavors,  any  plan  to  deploy  PBL  as  a  viable 
product  support  strategy  should  include  the  warfighters  and  their  critical 
perspectives  if  PBL  is  to  be  successfully  implemented. 

PBL  Awareness/Training  and  Incentives/Awards  should  be 
considered  effective  enablers  to  PBL  implementation  and  need  to  be  fully 
embraced  by  the  Services  and  the  Office  of  the  Secretary  of  Defense.  The 
2011  survey  indicated  that  current  respondents  consider  these  factors 
(Figure  1)  to  be  vital  to  the  success  of  PBL.  They  were  rated  as  barriers  in 
2005;  conversely,  they  have  been  shown  in  the  2011  study  to  be  effective 
enablers,  and  need  to  be  leveraged  as  such.  Continued  attention  should 
also  be  placed  on  ensuring  that  incentive-based  contracts  are  properly 
managed  by  DoD  and  the  Services’  contracting  agencies.  PBL  training 
should  also  be  continued  through  development  of  additional  courses 
and  Continuous  Learning  Modules  by  DAU  and  other  DoD  training 
agencies.  Senior  leaders  should  also  attend  similar  courses  and  related 
conferences/symposiums— especially  in  light  of  the  Cultural  Paradigms 
factor  discussed  next,  which  was  considered  to  be  the  most  significant 
PBL  barrier. 

Cultural  Paradigms  should  be  addressed  very  carefully  by  all  PBL 
implementers.  This  factor  was  identified  by  respondents  as  the  major 
barrier  to  successful  PBL  deployment.  DoD  leadership  must  address 
this  fact  and  ensure  that  PBL  training  is  provided  so  all  involved  under¬ 
stand  more  clearly  what  is  at  stake  (more  affordable  product  support, 
increased  readiness,  or  enhanced  efficiencies).  Additionally,  they  should 
understand  what  the  cultural  impediments  are  to  PBL’s  acceptance  as 
an  effective  means  to  ensure  greater  product  support  and  mission  effec¬ 
tiveness.  Continued  promulgation  of  success  stories  that  highlight  the 
true  capabilities  of  PBL  should  be  shared  throughout  the  Services— along 
with  how  and  who  have  been  most  successful  in  implementing  PBL.  Spe¬ 
cific  attention  should  be  placed  on  removing  cultural  impediments.  In 
particular,  future  training  should  include  awareness  of  related  cultural 
impediments  and  techniques  for  reducing  these  impediments.  The  target 
audience  for  this  type  of  training  would  be  senior  program  managers, 
systems  engineers,  contract  specialists,  and  logisticians. 
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Additional  emphasis  should  be  placed  on  enhancing  all  the  iden¬ 
tified  10  enablers.  Conversely,  efforts  should  be  placed  on  reducing  the 
effects  of  the  5  identified  barriers.  Besides  the  Warfighters’  Perspec¬ 
tives,  the  policy  responses  to  five  other  factors  (Strategic  Alliances/ 
Partnerships,  Supply  Chain  Management,  Performance  Metrics, 
TLCSM,  and  PB  Contracting)  are  likely  to  yield  large  benefits.  Besides 
focusing  on  Cultural  Paradigms,  the  four  grouped  items  identified  as  bar¬ 
riers  (Funding,  TD  Rights,  Existing  Infrastructure  or  Organization,  and 
Statutory-Regulatory  Requirements)  should  be  treated  as  opportunities 
for  mitigation  efforts  to  reduce  their  undesired  effects  on  successful  PBL 
implementation. 


Conclusions 

The  2011  survey  identified  10  critical  PBL  enablers  that  should  be 
enhanced;  it  also  identified  5  barriers  that  should  be  minimized  in  PBL 
implementation.  The  results  of  this  study  have  applications  to  successful 
implementation  of  PBL  throughout  DoD  and  the  commercial-industrial 
workplace. 

To  restate,  the  research  provided  the  following  results: 

•  The  single  most  significant  enabling  factor  for  PBL  was 
maintaining  the  Warfighters’  Perspectives  in  the  2011  study. 

•  The  top  barrier  to  PBL  was  Cultural  Paradigms  in  the  2011 
survey. 

•  Warfighters’  Perspectives  (2011)  replaced  Performance 
Metrics  from  the  2005  study  as  the  most  significant  enabler. 

•  Cultural  Paradigms  (2011)  replaced  Funding  from  the  2005 
study  as  the  most  significant  barrier. 

•  Two  barriers  from  the  2005  study  were  determined  to  be 
enablers  in  the  2011  study  (PBL  Awareness/Training  and 
Incentives/Awards). 
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Future  research  similar  to  this  effort  should  ensure  or  include  the 
participation  of  other  disciplines  such  as  systems  engineers,  testers,  con¬ 
tract  specialists,  and  business  cost  estimators  and  financial  managers, 
to  view  perceptions  about  PBL  through  a  “larger  aperture”  or  perspec¬ 
tive.  Additionally,  analyses  should  be  conducted  with  the  existing  data 
to  determine  if  survey  respondents  expressed  different  perceptions 
between  functional  career  areas,  e.g.,  do  all  supply  specialists  have  a 
different  perception  of  the  effect  training  has  on  PBL  implementation  vs. 
program  managers?  It  should  also  be  highlighted  that  the  survey  did  not 
specifically  question  respondents  on  whether  their  perceptions  of  PBL 
implementation  were  reflecting  their  opinions  on  each  factor’s  effect 
on  PBL  implementation  under  current  practices  and  policies,  or  if  their 
opinions  reflected  each  factor’s  effect  regardless  of  current  practices. 
This  is  an  important  consideration  when  one  is  considering  the  causes 
of  implementing  PBL.  Is  implementation  affected  more  by  specific  pro¬ 
gram  or  project-unique  PBL  policies  or  practices,  or  by  the  general  DoD/ 
Office  of  the  Secretary  of  Defense  policies  and  procedures  effects  on  PBL? 
Clearly,  more  research  is  required  on  the  current  dynamic  topic  of  PBL . 
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Running  With  Scissors: 

Defense  Budget  Cuts  and 
Potential  Industry  Responses 


Bryan  A.  Riley 


Nearly  everyone  can  relate  to  the  experience  of  seeing  a 
dangerous  sequence  of  events  unfold.  A  well-intentioned 
action  is  followed  by  a  subtle  misstep.  Add  in  a  measure 
of  unpredictability,  and  quickly  the  sequence  starts  to 
diverge.  In  these  situations,  a  reasonable  person  mentally 
fast-forwards  to  anticipate  the  possible  outcome.  It  is  that 
quick  mind’s  eye  picture  that  spurs  action.  It  prompts 
intervention.  Building  on  the  analysis  and  recommen¬ 
dations  presented  in  this  article,  the  author  makes  the 
case  that  it  is  possible  for  both  the  U.S.  Department 
of  Defense  and  the  U.S.  defense  industry  to  mitigate 
the  dangerous  downside  risk  of  anticipated  defense 
budget  cuts. 
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Nearly  everyone  can  relate  to  the  experience  of  seeing  a  dangerous 
sequence  of  events  begin  to  unfold.  A  well-intentioned  action  is  followed 
by  a  subtle  misstep.  Add  in  a  measure  of  unpredictability,  and  quickly 
the  sequence  starts  to  diverge.  In  these  situations,  a  reasonable  person 
mentally  fast-forwards  to  anticipate  the  possible  outcome.  It  is  that  quick 
mind’s  eye  picture  that  spurs  action.  It  prompts  intervention. 

As  unexpected  as  it  may  appear,  recent  debates  on  the  U.S.  defense 
budget  hold  many  of  these  same  concerns.  Reducing  federal  deficit 
spending  is  a  well-intentioned  action,  but  the  method  of  achieving  that 
objective  is  a  source  of  great  risk.  A  subtle  misstep  and  the  sequence  of 
events  could  quickly  accelerate— with  the  cascading  effects  growing 
beyond  even  the  most  diligent  efforts  to  avoid  a  wildly  negative  outcome. 

To  help  develop  this  thought,  the  following  discussion  addresses  six 
key  questions. 

•  Where  are  we  in  the  sequence  of  events? 

•  What  is  likely  to  happen  next? 

•  How  will  industry  respond? 

•  What  are  the  impacts  on  acquisition? 

•  Are  specific  scenarios  already  in  motion? 

•  What  can  be  done? 

Within  this  context,  the  thesis  of  this  article  is  simply  stated.  Even 
though  current  U.S.  defense  budgets  remain  strong,  actions  by  the  U.S. 
defense  industry  in  response  to  anticipated  spending  cuts  may  result  in 
increased  risk,  decreased  readiness,  and  ultimately  degradation  of  the 
nation’s  defense  industry  at  large. 

It  is  this  potentially  dangerous  scenario  that  characterizes  the  cur¬ 
rent  discussion  of  expected  U.S.  defense  budget  cuts.  Intervention  is 
possible.  First,  stop  running— then  slowly  put  down  the  scissors. 
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Where  Are  We  in  the  Sequence  of  Events? 

Current  U.S.  federal  deficit  spending  is  well  documented— as  is  the 
rapidly  increasing  federal  debt.  In  developing  a  baseline  for  the  discus¬ 
sion  of  defense  budget  reductions,  a  short  summary  of  past,  current,  and 
anticipated  federal  spending  is  useful  and  informative.  To  this  end,  the 
following  section  outlines  the  key  issues  of  discretionary  federal  expen¬ 
ditures,  defense  spending,  and  military  procurement  budgets. 

In  2010,  defense  expenditures  accounted  for  slightly  more  than  half 
of  federal  discretionary  spending  at  $689  billion,  including  the  cost  of 
overseas  contingency  operations  (OCO)  in  Iraq  and  Afghanistan  (Con¬ 
gressional  Budget  Office  [CBO],  2011b,  p.  69).  The  January  2011  baseline 
CBO  data  (Figure  1)  predicted  a  decline  in  both  defense  and  nondefense 
discretionary  expenditures  starting  in  2010,  and  continuing  through 
2021  as  a  percentage  of  Gross  Domestic  Product  (GDP).  Based  on  this 
forecast,  discretionary  spending  was  anticipated  to  fall  from  9.3  percent 
of  total  U.S.  GDP  in  2010  to  6.7  percent  by  2021  (CBO,  2011b,  p.  71). 

Using  January  2011  data  for  defense  spending  (Figure  1),  the  CBO 
forecasted  base  budget  growth  in  2011  dollars  continuing  through  2028 
with  an  anticipated  expenditures  forecast  using  the  current  Future  Years 
Defense  Program  (FYDP)  data  shown  in  Figure  2.  The  CBO  analysis 
included  forecasts  for  the  five  primary  budget  categories  of  defense 
spending:  (a)  military  construction,  (b)  research,  development,  test,  and 
evaluation,  (c)  procurement,  (d)  military  personnel,  and  (e)  operations 
and  maintenance. 


FIGURE  1.  DISCRETIONARY  SPENDING  (1971-2021) 


Source:  Congressional  Budget  Office  (as  of  January  2011). 
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FIGURE  2.  COSTS  OF  DoD'S  PLANS  FOR  ITS  BASE  BUDGET, 
BY  APPROPRIATION  CATEGORY 
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Notes.  Adapted  from  "Reducing  the  Deficit:  Spending  and  Revenue  Options,"  published  by  the 
Congressional  Budget  Office,  March  2011.  Each  category  shows  the  CBO  projection  of  the  base 
budget,  which  incorporates  costs  that  are  consistent  with  the  Department  of  Defense  (DoD)'s 
past  experience.  The  amounts  shown  for  the  FYDP  and  the  extension  of  the  FYDP  are  the  totals 
for  all  categories.  Base-budget  data  include  supplemental  funding  prior  to  2002.  FYDP  period 
=  2011  to  2015,  the  years  for  which  DoD’s  plans  are  fully  specified. 

*The  extension  of  the  FYDP  extends  DoD's  plans  and  uses  DoD's  estimates  of  costs  if  they 
are  available  and  cost  factors  based  on  the  broader  U.S.  economy  if  estimates  by  DoD  are  not 
available. 


In  February  2011,  the  CBO  reported  U.S.  defense  acquisition  spend¬ 
ing  from  1980  and  forecasted  anticipated  defense  acquisition  spending 
through  2028.  According  to  the  CBO,  2011  acquisition  costs  would 
account  for  34  percent  of  total  defense  spending  (excluding  OCO  costs). 
Based  on  the  current  FYDP,  the  CBO  forecasted  defense  acquisition  will 
grow  from  its  2011  level  of  $189  billion  to  $218  billion  in  2017  before 
starting  a  modest  decline  (CBO,  2011b,  p.  19)  as  indicated  in  Figure  3. 
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FIGURE  3.  COSTS  OF  DoD'S  BASE  BUDGET  ACQUISITION  PLANS, 
BY  DoD  COMPONENT 


300 


250 


50 


Notes.  Adapted  from  "Long-term  Implications  of  the  2011  Future  Years  Defense  Program," 
published  by  the  Congressional  Budget  Office,  February  2011.  Each  category  shows  the  CBO 
projection  of  the  base  budget,  which  incorporates  costs  that  are  consistent  with  the  Department 
of  Defense  (DoD)'s  past  experience.  Base-budget  data  include  supplemental  funding  prior  to 
2002.  The  amounts  shown  for  the  FYDP  and  the  extension  of  the  FYDP  are  the  totals  for  all 
components.  FYDP  period  =  2011  to  2015,  the  years  for  which  the  DoD’s  plans  are  fully  specified. 
MDA  =  Missile  Defense  Agency. 

’The  extension  of  the  FYDP  extends  DoD's  plans  and  uses  DoD’s  estimates  of  costs  if  they 
are  available  and  cost  factors  based  on  the  broader  U.S.  economy  if  estimates  by  DoD  are  not 
available. 


Before  completing  this  background  discussion,  it  is  important  to 
note  two  additional  considerations.  First,  from  2001  through  2011, 
almost  $1.2  trillion  has  been  appropriated  for  the  cost  of  overseas  con¬ 
tingency  operations.  Significant  uncertainty  exists  in  forecasting  future 
operational  expenditures;  for  purposes  of  the  following  discussion,  OCO 
funding  is  not  included  unless  specifically  noted.  Second,  as  experienced 
in  2011,  the  fiscal  year  federal  budget  may  not  receive  timely  approval 
by  the  U.S.  Congress.  In  this  case,  through  a  series  of  continuing  resolu¬ 
tions,  defense  spending  is  limited  to  prior-year  funding  levels.  In  recent 
statements,  former  Secretary  of  Defense  Robert  Gates  warned  that  the 
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impact  of  continuing  resolutions  “may  soon  turn  into  a  crisis”  (Gates, 
2011).  For  this  discussion,  the  anticipated  effects  are  not  included  unless 
specifically  noted. 

By  starting  with  this  summary  of  federal  discretionary  spending,  it 
is  now  possible  to  better  characterize  defense  acquisition  spending  in  the 
context  of  future  budget  reductions.  This  serves  to  answer  the  opening 
question— where  are  we  in  the  sequence  of  events? 


What  is  Likely  to  Happen  Next? 

Building  on  the  federal  expenditure  summary  presented  in  the  previ¬ 
ous  section,  the  following  discussion  summarizes  a  set  of  budget  reviews 
and  policy  opinions  affecting  the  U.S.  federal  budget.  In  particular,  this 
outline  compares  recommendations  from  recent  congressional  commit¬ 
tees,  federal  agencies,  and  think-tank  organizations. 

In  November  of  2010,  former  Senate  Budget  Committee  Chairman 
Pete  Domenici  and  former  White  House  budget  director  Alice  Rivlin 
led  a  bipartisan  study  to  deliver  a  set  of  broad  recommendations  that 
addressed  federal  spending  as  well  as  government  revenues.  The  goal 
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of  their  recommendation  was  to  balance  the  federal  budget  by  2014  and 
stabilize  the  national  debt  below  60  percent  of  GDP  by  2020.  As  part 
of  the  strategy,  defense  spending  would  be  frozen  at  current  levels  for 
5  years  and  then  capped  at  the  GDP  growth  rate.  In  further  detail,  sav¬ 
ings  would  come  from  reductions  in  military  force  structure,  acquisition, 
intelligence  operations,  personnel  costs,  and  current  efficiency  efforts 
already  underway.  Shown  in  Figure  4,  the  net  effect  of  these  decisions 
would  reduce  defense  spending  to  approximately  3  percent  of  U.S.  GDP 
by  2020  (Domenici  &  Rivlin,  2010). 


FIGURE  4.  DEFENSE  DISCRETIONARY  BUDGET  AUTHORITY: 
BASELINE  v.  BIPARTISAN  PLAN 


Defense  Baseline  Budget  Authority  (grows  with  GDP,  troop  levels  reduced  to  30,000  by  2013) 
5-year  Hard  Freeze  (2012-2016) 


In  December  of  2010,  a  team  led  by  Senator  Alan  Simpson  and  former 
White  House  Chief  of  Staff  Erskine  Bowles  presented  their  recommen¬ 
dations.  The  team  developed  a  set  of  actions  to  generate  approximately 
$4  trillion  in  deficit  reductions  through  2020.  Their  strategy  focused  on 
balancing  the  federal  budget  by  2015  and  reducing  the  federal  debt  to  60 
percent  of  GDP  by  2023,  and  40  percent  by  2035.  Beginning  in  2012,  all 
discretionary  spending  would  be  capped  at  2011  levels.  Defense  spend- 
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*  Note:  Levels  will  be  set  by  the  current  Congress  rather  than  the  statutory  caps;  the  2012  levels  will  be  at  or  below  the  final  2011  levels. 
$1,098  represents  a  12-month  Continuing  Resolution  as  a  default,  but  the  actual  number  could  differ  significantly. 
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ing  would  be  combined  with  other  categories  of  related  discretionary 
spending  to  be  managed  and  reduced  as  a  broader  category  of  security 
spending  (National  Commission,  2010)  shown  in  Figure  5. 

In  March  2011,  the  CBO  released  a  set  of  105  options  to  help  guide 
the  discussion  of  deficit  reduction  through  potential  changes  to  federal 
spending  and  revenue  policies.  Options  addressed  both  mandatory  as 
well  as  discretionary  spending  with  significant  detail  devoted  to  federal 
revenue  and  tax  policy.  On  defense  expenditures,  the  CBO  developed 
three  potential  scenarios  beginning  in  2012:  (a)  limit  growth  in  defense 
spending  to  1.4  percent  per  year  to  realize  a  reduction  of  $286  billion  by 
2021,  (b)  freeze  defense  spending  at  2011  levels  to  generate  $611  billion 
in  savings  by  2021,  and  (c)  reduce  defense  spending  by  1  percent  annually 
from  2011  levels  to  achieve  $862  billion  in  reductions  by  2021.  Looking 
at  options  that  focus  specifically  on  acquisition  programs,  the  CBO  out¬ 
lined  potential  reductions  of  $38  billion  by  2016  through  cancellations, 
deferrals,  and  force  reductions  (CBO,  2011a). 

Next,  turning  to  the  think-tank  groups,  a  wide  range  of  assessments 
and  recommendations  exists.  The  Center  for  Strategic  and  International 
Studies  echoed  the  concerns  raised  in  the  prior  sections  and  argued 
that  the  coming  decline  in  defense  spending  will  require  a  much  more 
involved  strategic  threat  assessment  to  establish  military  funding  pri¬ 
orities  (Berteau,  2009).  The  Center  for  Strategic  Budgetary  Assessments 
emphasized  the  need  for  the  U.S.  Department  of  Defense  to  respond  to 
the  upcoming  levels  of  budget  austerity  by  “accepting  some  risks  and 
divesting  of  lower  priority  programs  and  capabilities”  (Harrison,  2011). 
As  expected,  additional  recommendations  spanned  the  range  of  conser¬ 
vative,  moderate,  and  liberal  advocacy  consistent  with  groups  such  as  the 
Cato  Institute,  Heritage  Foundation,  Brookings  Institute,  Teal  Group, 
and  the  Center  for  American  Progress. 

From  this  brief  summary,  contemporary  thinking  points  to  a  con¬ 
sensus  that  current  levels  of  federal  spending  are  not  sustainable.  In 
particular,  there  is  a  consistent  emphasis  on  the  need  to  curb  discre¬ 
tionary  spending— defense  expenditures  in  particular.  This  forms  the 
response  to  the  second  key  question— what  is  likely  to  happen  next? 
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How  Will  Industry  Respond? 

In  an  uncertain  environment,  industry  is  likely  to  respond  by  reduc¬ 
ing  investments,  diversifying  its  market  base,  and  restructuring  its 
business  operations. 

First,  when  at  risk  of  declining  future  demand,  business  can  respond 
quickly  by  reducing  investment  in  two  categories:  capital  expenditures 
and  research  and  development.  These  actions  are  often  viewed  by  inves¬ 
tors  as  positive  near-term  strategies  since  cost  avoidance  is  typically 
realized  on  the  company’s  balance  sheet  as  improved  operating  margins 
and/or  dividends  are  returned  to  shareholders.  Unfortunately,  severe 
reductions  in  either  capital  investment  or  company-funded  research 
and  development  are  not  sustainable  in  a  competitive  business  environ¬ 
ment.  The  long-term  negative  impact  is  felt  by  both  the  company  and 
its  customers. 

Reasonable  levels  of  capital  investment  are  necessary  to  sustain  the 
infrastructure  and  systems  required  for  manufacturing  and  operating 
activities.  In  most  cases,  gaining  efficiencies  and  realizing  cost  savings 
require  increasing  levels  of  capital  investment.  The  business  case  for 
these  investments  requires  significant  long-term  returns  to  justify  the 
expenditures.  In  an  uncertain  environment,  the  business  case  rationale 
often  does  not  support  increasing  investment.  Moreover,  capital  expen¬ 
ditures  become  more  difficult  to  justify,  and  less  capital  is  invested  in 
the  business.  In  the  case  of  ongoing  production,  this  scenario  results  in 
increasing  cost  pressure  as  facilities  and  equipment  continue  to  age,  and 
support  systems  are  not  updated  with  improved  processes  or  technology. 

Similar  to  capital  investment,  company-funded  research  and  devel¬ 
opment  is  likely  to  decline.  While  the  impact  of  capital  investment  is 
more  visible,  the  impact  of  research  and  development  is  less  tangible. 
With  confidence  in  anticipated  demand  and  future  requirements,  indus¬ 
try  invests  in  the  development  of  technologies  to  compete  for  upcoming 
contracts.  Absent  that  future  opportunity,  businesses  will  not  emphasize 
research  and  development  over  other,  more  pressing  financial  needs. 
This  may  result  in  a  near-term  benefit  to  the  industry  as  resources  are 
applied  in  other  areas,  but  the  long-term  impact  to  technology  develop¬ 
ment  can  be  severe.  With  declining  investment,  future  capabilities  will 
require  longer  development  timelines.  Longer  timelines  introduce  both 
cost  and  performance  risk.  Viewed  together,  these  factors  combine  to 
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form  a  growing  disincentive  to  launch  new  programs  that  require  sig¬ 
nificant  advances  in  research  and  development.  Much  like  reductions 
in  capital  investment,  the  cycle  quickly  begins  to  develop  into  a  strongly 
negative  feedback  loop. 

Second,  industry  will  likely  respond  by  aggressively  seeking  to 
diversify  both  its  customer  and  product  base.  While  the  leading  U.S. 
defense  companies  are  involved  with  contracts  that  span  across  Service 
branches,  much  of  the  defense  industry  is  focused  on  providing  specific 
capabilities,  systems,  and  technologies  that  serve  a  very  narrow  customer 
group.  For  those  companies,  the  downside  risk  is  significant  as  budget 
cuts  take  place.  To  counter  this,  companies  will  seek  to  diversify  across 
Service  branches  as  well  as  outside  of  the  Defense  Department,  and 
into  other  areas  such  as  the  State  Department  and  Homeland  Security. 
However,  the  most  likely  diversification  opportunity  lies  in  competing 
for  potential  Foreign  Military  Sales  (FMS).  As  U.S.  military  forces  draw 
down  from  current  combat  operations,  the  stated  objective  is  to  develop 
the  capacity  of  foreign  governments  to  provide  for  their  own  security  and 
defense.  This  marks  a  clear  opportunity  for  the  U.S.  defense  industry  to 
diversify  its  customer  base  through  FMS. 

In  an  environment  of  declining  defense  spending,  U.S.  industry  will 
also  seek  to  diversify  its  products  and  services  offerings.  As  demand  has 
grown  quickly  over  the  past  decade,  some  defense  companies  have  devel¬ 
oped  into  providers  of  specialized  products  or  services.  This  pattern  is 
neither  unnatural  nor  unhealthy  in  the  short-term,  but  it  does  introduce 
sizeable  downside  risk  as  demand  begins  to  decline.  One  example  of 
this  pattern  is  the  result  of  rapid  growth  in  demand  for  unmanned  sys¬ 
tems.  While  many  providers  have  maintained  abroad  set  of  capabilities 
for  aircraft,  ground,  and  underwater  systems,  the  demand  for  specific 
capabilities  has  narrowed  the  market  base.  In  addition,  during  a  period 
of  increasing  demand,  some  companies  across  the  defense  industry  have 
struggled  to  leverage  opportunities  to  capture  services  and  support  con¬ 
tracts  while  quickly  expanding  to  meet  production  contracts.  Declining 
production  demand  will  free-up  capacity  that  could  be  applied  toward 
future  services  and  support  opportunities. 

Third,  the  U.S.  defense  industry  will  look  to  restructure  itself  as 
defense  budgets  decline.  During  the  industry  downturn  in  the  early  1990s 
after  the  collapse  of  the  Soviet  Union,  the  U.S.  defense  industry  saw  a 
number  of  large-scale  consolidations.  Northrop  merged  with  Grumman 
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Aerospace,  Lockheed  with  Martin  Marietta,  and  Boeing  with  McDon¬ 
nell  Douglas.  Other  companies  divested  some  of  their  core  operations 
such  as  General  Dynamics’  divestiture  of  its  military  aircraft  business 
to  Lockheed  Martin.  Others  such  as  Raytheon  purchased  lower  tier 
companies  including  Hughes  and  E-Systems  to  build  a  much  broader 
business  portfolio  centered  around  systems  technology.  This  particular 
industry  response  will  be  fueled  by  the  recent  economic  recovery  leaving 
companies  with  sizeable  cash  reserves. 

As  described  in  this  section,  industry  is  likely  to  react  to  anticipated 
reductions  in  defense  spending  by  reducing  investment,  diversifying  its 
business  base,  and  restructuring  its  operations.  This  characterizes  the 
response  to  the  third  question— how  will  industry  respond? 

What  Are  the  Impacts  on  Acquisition? 

The  impact  of  anticipated  reductions  in  U.S.  defense  spending  will 
be  seen  in  acquisition  scope,  structure,  and  competition. 

First,  the  scope  of  acquisition  will  change.  Driven  by  high  operational 
tempo  duringthe  past  10  years,  U.S.  military  acquisition  has  focused  on 
satisfying  the  needs  of  the  ongoing  conflicts  in  Iraq  and  Afghanistan. 
New  weapon  systems  have  been  developed  and  deployed  such  as  the 
unmanned  Predator  drone  and  the  Mine-Resistant  Ambush  Protected 
vehicle.  Declining  budgets  will  shift  the  focus  of  defense  acquisition 
away  from  new  capabilities  to  more  modest  upgrades  and  derivatives 
of  weapon  systems  within  the  current  U.S.  military  inventory.  Support 
and  services  contracts  will  transition  to  organic  resources  as  U.S.  forces 
draw  down  from  their  current  deployments.  However,  the  most  severe 
acquisition  impacts  are  likely  to  come  from  program  deferrals  through 
reduced  production  rates,  reductions  through  decreased  unit  quantities, 
and— in  the  extreme  case— program  terminations. 

Second,  the  structure  of  acquisition  will  change.  Expanding  defense 
budgets  have  accommodated  a  wide  range  of  acquisition  structures, 
from  single-year  fixed-price  contracts  for  combat  support  and  supplies 
to  large  scale  cost-reimbursable  development  programs.  As  defense 
budgets  decline,  pressures  will  increase  to  reduce  acquisition  costs  by 
restructuring  existing  contracts.  Multiyear  contracts  will  transition 
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to  single-year,  cost-plus  contracts  will  convert  to  fixed-price,  and  block 
quantity  purchases  will  reissue  as  individual  unit  quantity  contracts 
with  priced  options.  While  this  approach  is  likely  to  meet  the  near-term 
goal  of  reducing  current  year  acquisition  costs,  the  net  effect  will  likely 
drive  increasing  unit  costs.  This  scenario  risks  sparking  the  dangerous 
iteration  loop  experienced  in  recent  years  where  budget  pressures  force 
reductions  in  quantities  and/or  production  rates,  which  in  turn  drive 
higher  unit  costs.  History  has  been  particularly  unkind  to  programs 
caught  in  this  acquisition  scenario. 

Third,  the  competition  for  acquisition  will  change.  During  the  recent 
environment  of  increasing  demand,  the  limiting  constraint  has  often 
been  industry  capacity.  For  U.S.  defense  companies,  this  has  created  a 
very  attractive  advantage  where  demand  in  many  cases  exceeded  supply. 
However,  as  demand  declines,  the  result  will  likely  be  improved  pricing 
for  the  U.S.  military  as  companies  begin  to  trade  operating  margins  for 
continued  revenues.  This  pattern  will  be  particularly  strong  in  product 
classes  requiring  high  fixed  costs  and  significant  capital  investments 
such  as  aircraft  and  shipbuilding.  In  some  cases,  the  growing  pressures 
will  result  in  companies  divesting  unprofitable  operations  or  even  choos¬ 
ing  to  close  down  certain  business  lines.  Compounding  this  effect,  global 
defense  budgets  are  also  expected  to  decline.  As  international  demand 
decreases,  foreign  competition  for  U.S.  military  acquisition  will  inten¬ 
sify.  EADS,  BAE,  and  others  will  increase  their  already  growing  search 
for  opportunities  in  the  U.S.  defense  market. 

So  for  acquisition,  three  areas  of  impact  are  most  likely:  scope, 
structure,  and  competition.  These  categories  help  address  the  fourth 
question— what  are  the  impacts  on  acquisition? 

Are  Specific  Scenarios  Already  in  Motion? 

Applying  the  concepts  developed  throughout  the  previous  sections,  it 
is  now  possible  to  develop  a  case  study  using  the  top  largest  U.S.  defense 
companies:  Lockheed  Martin,  Boeing  Defense,  Northrop  Grumman, 
General  Dynamics,  and  Raytheon.  To  establish  a  baseline  comparison, 
the  following  discussion  uses  data  compiled  from  U.S.  Securities  and 
Exchange  Commission  10-K  Reports  for  the  2001  to  2010  reporting 
periods,  summarized  in  Figure  6. 
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FIGURE  6.  U.S.  SECURITIES  &  EXCHANGE  COMMISSION  10-K  REPORTS 
(2001-2010  REPORTING  PERIODS) 


2001 

2002 

2003 

General  Dynamics 

Annual  Revenue 

$B,USD 

$23,990 

$26,578 

$31,824 

Annual  Operating  Earnings 

$B,USD 

$0,833 

$1,158 

$2,019 

Annual  Operating  Margin 

%  Annual  Revenue 

3.5% 

4.4% 

6.3% 

Capital  Expenditure 

%  Annual  Revenue 

- 

2.5% 

2.2% 

Research  &  Development 

%  Annual  Revenue 

- 

- 

- 

Boeing  (Defense) 

Annual  Revenue 

$B,USD 

$22,815 

$24,957 

$27,361 

Annual  Operating  Earnings 

$B,USD 

$1,965 

$2,009 

$0,766 

Annual  Operating  Margin 

%  Annual  Revenue 

8.6% 

8.0% 

2.8% 

Capital  Expenditure 

%  Annual  Revenue 

2.6% 

2.2% 

1.4% 

Research  &  Development 

%  Annual  Revenue 

2.3% 

2.0% 

2.1% 

Northrop  Grumman 

Annual  Revenue 

$B,USD 

$13,558 

$17,206 

$26,206 

Annual  Operating  Earnings 

$B,USD 

$1,004 

$1,391 

$1,538 

Annual  Operating  Margin 

%  Annual  Revenue 

7.4% 

8.1% 

5.9% 

Capital  Expenditure 

%  Annual  Revenue 

2.9% 

3.1% 

2.4% 

Research  &  Development 

%  Annual  Revenue 

2.5% 

1.6% 

1.6% 

General  Dynamics 

Annual  Revenue 

$B,USD 

$12,163 

$13,829 

$16,617 

Annual  Operating  Earnings 

$B,USD 

$1,485 

$1,582 

$1,467 

Annual  Operating  Margin 

%  Annual  Revenue 

12.2% 

11.4% 

8.8% 

Capital  Expenditure 

%  Annual  Revenue 

2.9% 

1.9% 

1.3% 

Research  &  Development 

%  Annual  Revenue 

1.7% 

1.8% 

1.7% 

Raytheon 

Annual  Revenue 

$B,USD 

$16,867 

$16,760 

$18,109 

Annual  Operating  Earnings 

$B,USD 

$0,759 

$1,754 

$1,316 

Annual  Operating  Margin 

%  Annual  Revenue 

4.5% 

10.5% 

7.3% 

Capital  Expenditure 

%  Annual  Revenue 

2.7% 

2.7% 

2.4% 

Research  &  Development 

%  Annual  Revenue 

2.7% 

2.7% 

2.7% 
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2004 

2005 

2006 

2007 

2008 

2009 

2010 

General  Dynamics  continued 

$35,526 

$37,213 

$39,620 

$41,862 

$42,731 

$45,189 

$45,803 

$2,089 

$2,986 

$3,953 

$4,527 

$5,131 

$4,466 

$4,097 

5.9% 

8.0% 

10.0% 

10.8% 

12.0% 

9.9% 

8.9% 

2.2% 

2.3% 

2.3% 

2.2% 

2.2% 

1.9% 

1.8% 

- 

1.7% 

1.6% 

1.6% 

1.6% 

1.6% 

1.4% 

Boeing  (Defense)  continued 

$30,465 

$30,791 

$32,439 

$32,080 

$32,047 

$33,661 

$31,943 

$2,925 

$3,890 

$3,032 

$3,440 

$3,232 

$3,299 

$2,875 

9.6% 

12.6% 

9.3% 

10.7% 

10.1% 

9.8% 

9.0% 

0.7% 

1.4% 

1.0% 

0.7% 

0.8% 

0.8% 

0.7% 

1.8% 

1.9% 

1.7% 

1.7% 

1.9% 

2.3% 

2.4% 

Northrop  Grumman  continued 

$29,853 

$30,721 

$30,148 

$32,018 

$33,887 

$33,755 

$34,757 

$2,006 

$2,178 

$2,454 

$3,006 

-$0,111 

$2,483 

$3,070 

6.7% 

7.1% 

8.1% 

9.4% 

-0.3% 

7.4% 

8.8% 

2.3% 

2.7% 

2.4% 

2.1% 

2.0% 

1.9% 

2.2% 

1.7% 

1.7% 

1.9% 

1.7% 

1.7% 

1.8% 

1.7% 

General  Dynamics  continued 

$19,178 

$21,244 

$24,063 

$27,240 

$29,300 

$31,981 

$32,466 

$1,941 

$2,197 

$2,625 

$3,113 

$3,653 

$3,675 

$3,945 

10.1% 

10.3% 

10.9% 

11.4% 

12.5% 

11.5% 

12.2% 

1.4% 

1.2% 

1.4% 

1.7% 

1.7% 

1.2% 

1.1% 

1.7% 

1.6% 

1.6% 

1.6% 

1.6% 

1.6% 

1.6% 

Raytheon  continued 

$20,245 

$21,894 

$20,291 

$21,301 

$23,174 

$24,881 

$25,183 

$1,388 

$1,687 

$1,840 

$2,328 

$2,596 

$3,042 

$2,607 

6.9% 

7.7% 

9.1% 

10.9% 

11.2% 

12.2% 

10.4% 

1.8% 

1.4% 

1.4% 

1.5% 

1.3% 

1.1% 

1.3% 

2.4% 

2.0% 

2.3% 

2.4% 

2.2% 

2.3% 

2.5% 
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FIGURE  7.  REVENUES  OF  TOP  U.S.  DEFENSE  FIRMS  OVER 
LAST  DECADE 
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FIGURE  8.  COMPARISON  OF  ANNUAL  OPERATING  MARGINS 
OF  TOP  U.S.  DEFENSE  FIRMS  OVER  LAST  DECADE 


Looking  first  at  revenues  in  Figure  7,  each  of  the  top  U.S.  Defense 
firms  experienced  significant  sales  growth  during  the  last  decade.  In 
particular,  General  Dynamics  and  Northrop  Grumman  saw  annual 
revenues  more  than  double. 

In  contrast  to  the  steady  increase  in  annual  revenues,  a  comparison 
of  annual  operating  margins  shown  in  Figure  8  is  more  unstable.  Two 
features  to  note:  In  2003,  Boeing-Defense  recorded  a  one-time  charge  of 
$1.7  billion  against  its  space  launch  and  orbital  systems  division,  which 
drove  the  company’s  performance  down  to  2.8  percent  for  the  year.  In 
2008,  Northrop  Grumman  announced  a  $2.5  billion  write-off  against  its 
shipbuilding  business  that  resulted  in  a  net  loss  of  -$111  million  for  the 
reporting  year  (The  Boeing  Company  et  al.,  2001-2010). 
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FIGURE  9.  ANNUAL  RESEARCH  AND  DEVELOPMENT 


EXPENDITURES,  PERCENTAGE  OF  ANNUAL  REVENUES 
FOR  TOP  FIVE  U.S.  DEFENSE  FIRMS  (2001-2010) 
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FIGURE  10.  ANNUAL  CAPITAL  EXPENDITURES,  PERCENTAGE 
OF  ANNUAL  REVENUES  FOR  TOP  FIVE  U.S.  DEFENSE  FIRMS 


(2001-2010) 
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Next,  applying  the  framework  developed  for  likely  industry 
responses,  it  is  helpful  to  compare  research  and  development  expendi¬ 
tures  along  with  capital  investments.  For  the  top  five  U.S.  defense  firms. 
Figures  9  and  10  show  the  data  from  2001  through  2010  as  a  percentage 
of  annual  revenues. 

From  the  data,  research  and  development  expenses  are  fairly  con¬ 
stant,  with  Boeing  Defense  indicating  a  noticeable  increase  in  investment 
over  the  past  three  reporting  years.  However,  capital  expenditures  for  the 
top  defense  companies  show  a  clear  downward  trend.  This  downward 
trend  is  led  by  Boeing  Defense,  the  same  company  that  indicated  stronger 
recent  investment  in  research  and  development. 
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FIGURE  11.  SEC  DATA  COMPILED  FOR  NORTHROP  GRUMMAN 
BUSINESS  SEGMENTS  (2001-2010) 


2001 

2002 

2003 

Northrop  Grumman  (Business  Segments) 

Annual  Revenue 

$B,USD 

$3.00 

$3.27 

$6.62 

Annual  Operating  Earnings 

$M,USD 

$258 

$331 

$573 

Annual  Operating  Margin 

%  Annual  Revenue 

8.60% 

10.10% 

8.70% 

Electronics 

Annual  Revenue 

$B,USD 

$4.72 

$5.34 

$6.04 

Annual  Operating  Earnings 

$M,USD 

$359 

$435 

$590 

Annual  Operating  Margin 

%  Annual  Revenue 

7.60% 

8.10% 

9.80% 

Information  Systems 

Annual  Revenue 

$B,USD 

$3.78 

$4.24 

$8.87 

Annual  Operating  Earnings 

$M,USD 

$170 

$249 

$539 

Annual  Operating  Margin 

%  Annual  Revenue 

4.50% 

5.90% 

6.10% 

Shipbuilding 

Annual  Revenue 

$B,USD 

$1.88 

$4.71 

$5.45 

Annual  Operating  Earnings 

$M,USD 

$19 

$306 

$295 

Annual  Operating  Margin 

%  Annual  Revenue 

1.00% 

6.50% 

5.40% 

Technical  Services  (Omitted  from  Comparison  Charts  - 
Segment  Operations  Started  in  2004) 

Annual  Revenue 

$B,USD 

Annual  Operating  Earnings 

$M,USD 

Annual  Operating  Margin 

%  Annual  Revenue 

While  this  simple  comparison  is  not  conclusive,  it  is  indicative  of 
the  industry  response  of  reduced  investment  presented  in  the  financial 
performance  data  of  the  top  five  U.S.  defense  companies. 

Next,  data  available  in  the  10-K  annual  reports  help  provide  exam¬ 
ples  of  the  need  for  diversification.  In  2010,  Lockheed  Martin  reported 
84  percent  of  the  company’s  net  revenues  from  the  U.S.  Government, 
with  15  percent  of  revenues  from  FMS.  Also  in  2010,  Boeing  Defense 
reported  87  percent  of  revenues  from  contracts  to  the  U.S.  Government, 
and  Northrop  Grumman’s  U.S.  Government  sales  were  92  percent  of 
2010  revenues.  General  Dynamics  reported  72  percent  of  revenues  from 
the  U.S.  Government,  and  Raytheon  reported  88  percent  of  total  sales 
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2004 

2005 

2006 

2007 

2008 

2009 

2010 

Northrop  Grumman  (Business  Segments)  continued 

$8.01 

$9.01 

$8.85 

$8.20 

$9.84 

$10.42 

$10.91 

$634 

$729 

$844 

$852 

$417 

$1,071 

$1,256 

7.90% 

8.10% 

9.50% 

10.40% 

4.20% 

10.30% 

11.50% 

Electronics  continued 

$6.42 

$6.64 

$6.58 

$6.91 

$7.09 

$7.67 

$7.61 

$670 

$710 

$744 

$813 

$952 

$969 

$1,023 

10.40% 

10.70% 

11.30% 

11.80% 

13.40% 

12.60% 

13.40% 

information  Systems  continued 

$10.00 

$10.62 

$9.11 

$10.42 

$10.16 

$8.61 

$8.40 

$622 

$736 

$823 

$895 

$813 

$631 

$756 

6.20% 

6.90% 

9.00% 

8.60% 

8.00% 

7.30% 

9.00% 

Shipbuilding  continued 

$6.25 

$5.79 

$5.32 

$5.79 

$6.15 

$6.21 

$6.72 

$389 

$241 

$393 

$538 

($2,307) 

$299 

$325 

6.20% 

4.20% 

7.40% 

9.30% 

-37.50% 

4.80% 

4.80% 

Technical  Services  (Omitted  from  Comparison  Charts  - 
Segment  Operations  Started  in  2004)  continued 

$0.23 

$0.04 

$1.79 

$2.18 

$2.30 

$2.78 

$3.23 

($3) 

($17) 

$110 

$120 

$121 

$161 

$206 

-1.30% 

-40.50% 

6.10% 

5.50% 

5.30% 

5.80% 

6.40% 

(The  Boeing  Company  et  al.,  2001-2010).  Just  looking  at  the  most  recent 
2010  reporting  period,  financial  performance  of  the  top  U.S.  defense 
companies  is  clearly  at  significant  risk  as  defense  budgets  decline.  Diver¬ 
sification  will  likely  be  a  key  industry  response. 

Turning  next  to  restructuring,  the  initial  set  of  comparisons  over 
the  past  10  years  ranks  Northrop  Grumman  as  third  in  annual  revenues 
and  last  in  annual  operating  margins.  Narrowing  the  focus,  performance 
data  for  each  of  Northrop  Grumman’s  business  segments  indicate  poten¬ 
tial  industry  responses.  Figure  11  summarizes  Northrop  Grumman’s 
business  segment  performance  from  2001-2010,  and  Figures  12  and  13 
compare  annual  revenues  and  operating  margins. 
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FIGURE  12.  ANNUAL  REVENUES  OF  NORTHROP  GRUMMAN'S 
BUSINESS  SEGMENTS  (2001-2010) 
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FIGURE  13.  ANNUAL  OPERATING  MARGINS  OF  NORTHROP 
GRUMMAN'S  BUSINESS  SEGMENTS  (2001-2010) 
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Based  on  these  comparisons,  Northrop  Grumman  shipbuilding  lags 
the  company’s  other  business  segments  in  both  revenues  and  operating 
margins,  with  shipbuilding  reflecting  the  sizeable  impact  of  the  $2.5  bil¬ 
lion  charge  taken  in  2008.  This  likely  helps  provide  important  context 
to  the  company’s  announcement  in  2010  that  it  intended  to  spin  off  its 
shipbuilding  business  (The  Boeing  Company  et  al.,  2001-2010).  While 
only  a  singular  example,  this  serves  to  highlight  the  active  industry 
response  of  restructuring. 

Using  the  information  developed  in  this  short  case  study,  the 
response  is  positive  to  the  fifth  key  question— are  specific  scenarios 
already  in  motion? 
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What  Can  Be  Done? 

Fortunately,  much  can  be  done  to  counter  the  downside  risk  of  a 
declining  U.S.  defense  budget.  The  following  section  outlines  several 
recommendations  to  mitigate  the  risk  to  the  U.S.  military  as  well  as  the 
U.S.  defense  industry. 

The  first  recommendation  is  to  enact  an  aggressive  incentive  pro¬ 
gram  focused  on  business  investment.  Current  plans  for  accelerated 
capital  depreciation  credit  align  very  well  with  this  recommendation. 
Since  the  financial  rationale  for  capital  expenditures  is  driven  by  how 
quickly  a  company  can  recover  its  investment,  an  accelerated  depre¬ 
ciation  credit  has  a  significant  positive  effect  through  taxation  benefit. 
In  addition,  an  increased  research  and  development  tax  credit  should 
also  be  enacted  to  better  align  financial  incentives  with  the  strategic 
benefit  of  investment  by  U.  S .  defense  companies  in  future  technologies. 
It  is  important  to  note  that  the  public  policy  dimension  of  this  recom¬ 
mendation  will  be  a  considerable  challenge.  Even  as  the  global  economy 
recovers,  U.S.  lawmakers  will  struggle  to  balance  the  strategic  benefits 
of  policies  such  as  these  with  the  need  to  sustain  or  increase  federal 
tax  revenues.  This  recommendation  will  help  counter  industry’s  likely 
response  of  reducing  investment. 

The  second  recommendation  addresses  potential  restructuring  of 
the  U.S.  defense  industry  by  updating  federal  policy  guidance  and  pro¬ 
cesses.  The  downturn  during  the  1990s  spurred  a  series  of  consolidation 
actions  across  the  defense  industry,  with  the  majority  of  these  activi¬ 
ties  resulting  in  mergers  or  acquisitions.  As  a  result,  the  overwhelming 
majority  of  today’s  federal  policies  focus  on  antitrust  and  competitive 
concerns.  Similarly,  reform  efforts  related  to  the  Committee  on  Foreign 
Investment  in  the  United  States  pushed  the  balance-point  further  away 
from  beneficial  foreign  ownership  of  businesses  that  provide  capabilities 
to  the  U.S.  military.  As  the  global  economy  has  changed  during  the  past 
decade,  these  policies  have  become  increasingly  out  of  date  (Department 
of  Defense,  1996).  Further  complicating  the  concern,  a  number  of  different 
federal  departments  and  agencies  are  responsible  for  areas  of  the  review 
and  approval  process.  From  the  Department  of  Justice  to  the  Securities 
and  Exchange  Commission,  no  clear  process  owner  exists.  In  addition  to 
updating  the  policies  involved,  clear  process  ownership  should  be  estab¬ 
lished.  This  recommendation  is  essential  to  successfully  managing  the 
likely  industry  response  of  restructuring  (Department  of  Defense,  2003). 
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The  third  recommendation  is  to  accelerate  current  reform  efforts 
related  to  U.S.  export  controls.  Much  like  the  industry  restructuring 
policies  outlined  previously,  military  export  controls  have  not  kept  pace 
with  changes  in  the  global  environment.  In  August  of  2009,  the  White 
House  initiated  a  review  to  identify  needed  export  reforms,  but  the  basic 
issues  and  struggles  remain.  Oversight  and  authority  for  export  controls 
continue  to  be  redundant  and— in  the  assessment  of  the  Government 
Accountability  Office  (GAO)— overly  restrictive  and  ineffective  (GAO, 
2010).  In  some  cases,  the  real  effect  on  U.S.  defense  companies  is  an 
inability  to  compete  for  FMS  opportunities,  while  European  or  Asian 
companies  expand  their  global  market  share.  As  the  U.S.  defense  indus¬ 
try  seeks  to  diversify  both  its  customer  and  product  base,  more  effective 
and  better  balanced  export  requirements  will  be  key  to  this  strategy. 

The  fourth  recommendation  emphasizes  the  importance  of  contin¬ 
ued  open  and  proactive  communication.  Through  formal  statements  and 
even  informal  remarks.  Department  of  Defense  officials  can  signal  to 
industry  what  actions  it  will  support  as  well  as  what  actions  it  will  not. 
As  an  example,  in  February  of  2011  former  Under  Secretary  of  Defense 
Ashton  Carter  delivered  timely  guidance  to  industry  by  stating  that 
the  Department  of  Defense  would  support  consolidation  of  second-  and 
third -tier  suppliers,  but  not  first-tier  defense  companies.  For  businesses 
diligently  working  to  develop  strategies  for  declining  defense  budgets, 
this  level  of  openness  is  essential  (Carter,  2011).  The  costs  to  evaluate 
and  formally  propose  a  potential  merger  or  acquisition  are  significant. 
The  least  favorable  outcome  is  for  industry  to  invest  the  resources  only 
to  have  the  federal  government  determine  that  the  proposal  is  not  in  its 
best  interest.  Clear,  open,  and  proactive  communication  is  key. 

In  summary,  this  set  of  recommendations  focuses  on  investment 
incentives,  industrial  review  policies,  export  reforms,  and  proactive 
communication.  Combined  together,  these  form  an  effective  response 
to  the  sixth  question— what  can  be  done? 
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Conclusions 

Mirroring  the  structure  developed  at  the  beginning  of  this  discus¬ 
sion,  the  following  section  addresses  each  of  the  opening  questions. 

Where  are  we  in  the  sequence  of  events?  It  would  be  comforting 
to  describe  how  the  United  States  is  still  very  early  in  the  timeline  and 
how  a  wide  range  of  options  remains  open  to  policy  makers.  However, 
the  reality  is  that  more  than  a  decade  of  federal  deficit  spending  and  the 
resulting  increase  in  the  national  debt  has  crossed  the  point  of  crisis. 
Simple  options  are  no  longer  available. 

What  is  likely  to  happen  next?  Prior  to  release  of  the  President’s 
2012  budget,  expectations  centered  on  the  need  for  a  series  of  strategic 
commitments  to  responsibly  draw  down  federal  expenditures  while 
increasing  revenues.  Now,  growing  consensus  opinion  points  to  a  near- 
term  scenario  requiring  dramatic  cuts  in  federal  spending  that  include 
sharp  reductions  in  discretionary  expenditures  focused  on  the  U.S. 
defense  budget. 

How  will  industry  respond?  Based  on  the  discussion,  three  likely 
scenarios  for  industry’s  response  emerge:  reduced  investment,  diver¬ 
sification,  and  restructuring.  Near-term  response  to  uncertainty  will 
impact  capital  investment  as  well  as  research  and  development  spending. 
Companies  will  also  pursue  opportunities  to  diversify  their  customer 
base  as  well  as  their  range  of  products  and  services.  As  industry  adjusts 
to  declining  future  demand,  many  businesses  will  choose  to  restructure 
through  acquisitions,  divestiture,  or  mergers.  In  some  cases,  companies 
may  choose  to  no  longer  compete  for  U.S.  defense  business. 

What  are  the  impacts  on  acquisition?  The  impact  of  declining 
U.S.  defense  budgets  will  be  seen  in  acquisition  scope,  structure,  and 
competition.  Scope  will  transition  from  a  focus  on  wartime  supply  and 
rapid  development  to  much  more  modest  upgrades  and  reset  activities. 
Structure  will  transition  from  a  broad  mix  of  contract  types  to  a  more 
narrow  set  of  shorter  duration,  fixed-price,  limited-quantity  contracts. 
Competition  will  increase  significantly  both  in  depth  and  breadth  as 
U.S.  companies  begin  to  trade  operating  margins  for  continued  rev¬ 
enues,  and  international  companies  increase  efforts  to  compete  for  U.S. 
defense  contracts. 
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Are  specific  scenarios  already  in  motion?  Researching  the  top 
five  U.S.  defense  firms,  elements  ofthe  predicted  industry  response  clearly 
exist.  Investment  in  areas  such  as  capital  expenditure  is  in  decline.  In  par¬ 
ticular,  Boeing  Defense  has  reduced  its  rate  of  capital  investment  by  over 
half  during  the  past  decade.  Diversification  is  underway  as  companies 
compete  for  FMS  opportunities.  Many  examples  of  restructuring  exist 
to  include  acquisitions,  divestitures,  and  proposed  mergers. 

What  can  be  done?  Four  policy  recommendations  hold  particular 
promise.  First,  the  federal  government  should  provide  financial  incentives 
for  business  investment  through  increased  tax  credit  for  research  and 
development  as  well  as  capital  investments.  Second,  the  policies  that  gov¬ 
ern  the  restructuring  of  U.S.  defense  companies  should  be  consolidated 
and  updated  to  include  beneficial  foreign  investment.  Third,  the  current 
effort  to  revise  U.S.  export  controls  should  be  accelerated  to  enable  market 
diversification  of  U.  S.  defense  companies  through  foreign  military  sales. 
Fourth,  the  Department  of  Defense  should  increase  its  use  of  industry 
forums,  public  statements,  and  other  communication  channels  to  signal 
its  intentions  as  companies  develop  their  response  strategies. 

Closing  Comments 

Building  on  the  analysis  and  recommendations  presented  in  this 
discussion,  it  is  possible  for  both  the  U.S.  Department  of  Defense  and 
the  U.S.  defense  industry  to  mitigate  the  dangerous  downside  risk  of 
anticipated  defense  budget  cuts. 

Intervention  is  possible. 

Now,  stop  running— and  slowly  put  down  the  scissors. 
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A  common  problem  in  defense  acquisition  is  the  diffi¬ 
culty  in  ensuring  that  the  required  capabilities  stated 
in  capability  development  documents  are  technically 
feasible,  affordable,  and  available  through  mature  tech¬ 
nologies.  This  problem  is  driven  by  a  lack  of  knowledge 
on  both  the  capability  developer  and  program  manager 
teams.  Addressing  this  knowledge  gap  requires  a  new 
approach  to  capability  development,  where  knowledge 
gained  early  in  the  process  is  injected  into  the  capability 
development  process  in  a  rigorous  way.  This  article 
describes  that  new  technical  approach  along  with  lessons 
learned  on  two  large  acquisition  programs.  Key  tenets 
include  the  use  of  pre-planned  knowledge  points  as  a 
vehicle  for  expanded  collaboration  between  program 
managers  and  capability  developers,  and  early  use  of 
systems  engineering  fundamentals. 
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The  current  capability  development  environment  includes  a  host  of 
challenges:  the  need  for  increasingly  capable  systems,  often  with  greater 
complexity;  time  constraints  and  the  resultant  pressure  on  rapid  delivery 
of  new  capabilities;  and  increased  cost  pressures.  As  new  capabilities 
are  developed,  the  threat  and  operational  environment  continues  to 
adapt,  often  necessitating  midstream  changes  to  requirements  or  other 
aspects  of  the  capability.  Mandatory  requirements  to  satisfy  larger  DoD 
policy  goals  must  also  be  addressed.  These  challenges  are  made  more 
difficult  by  a  lack  of  knowledge  on  the  part  of  the  capability  developer  as 
well  as  the  program  management  teams  regarding  technology  maturity, 
technical  feasibility,  and  affordability.  This  situation  makes  it  difficult 
to  reconcile  requirements  stated  in  capability  development  documents, 
with  the  'state  of  the  possible'  in  terms  of  feasibility  and  cost.  The  pur¬ 
pose  of  this  article  is  to  outline  a  technical  approach  to  addressing  these 
problems.  Key  tenets  of  this  approach  are  use  of  pre-planned  Knowledge 
Points  as  a  vehicle  for  expanded  collaboration  between  the  capability  and 
program  managers  (PM),  and  early  use  of  systems  engineering  funda¬ 
mentals  in  the  capability  development  process. 

This  approach  has  been  demonstrated  on  the  Joint  Light  Tactical 
Vehicle  (JLT  V)  program  throughout  the  Technology  Development  (TD) 
Phase  (over  36  months  until  its  Joint  Requirements  Oversight  Council 
[JROC]  approval),  and  based  on  that  success  is  now  being  implemented 
on  the  Marine  Corps  Amphibious  Combat  Vehicle  program.  Although 
these  programs  remain  in  development,  the  purpose  of  this  article  is  to 
describe  a  technical  approach  that  has  shown  promise  for  those  PMs 
opting  to  apply  the  techniques  and  lessons  learned  described  herein  to 
their  own  programs. 


Background 

In  September  2007,  then  Under  Secretary  of  Defense  for  Acquisi¬ 
tion,  Technology  and  Logistics  (USD[AT&L])  John  Young  directed 
that  all  acquisition  programs  requiring  USD(AT&L)  approval  include 
competitive,  technically  mature  prototyping  from  two  or  more  industry 
teams  through  Milestone  B.  Programs  requiring  USD(AT&L)  approval 
are  typically  the  largest,  most  expensive,  and  most  complex  (Young, 
2007).  Competitive  prototyping  was  later  incorporated  into  Depart¬ 
ment  of  Defense  Instruction  5000.02.  Secretary  Young  directed  this 
policy  to  address  the  problem  of  large  weapon  system  programs  being 
initiated  with  an  inadequate  understanding  of  technical  risk,  with- 
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out  firm  requirements,  and  with  a  weak  foundation  for  estimating 
developmental  and  procurement  costs.  This  situation  results  in  an 
unacceptable  number  of  programs  not  meeting  performance,  cost,  or 
schedule  requirements.  The  JLTV  program  was  the  first  ACAT  ID  pro¬ 
gram  to  apply  this  directive.  This  competitive  prototyping  paradigm 
in  the  TD  phase  offers  capability  developers  a  unique  opportunity,  but 
confers  a  responsibility  for  a  technically  sound  capability  development 
approach. 

As  foreshadowed  by  implementation  of  the  Joint  Capabilities  Inte¬ 
gration  and  Development  System  (JCIDS)  by  DoD  in  2003,  this  new 
approach  to  capability  development  involves  early  use  of  systems  engi¬ 
neering  and  technical  analyses  to  supplement  the  existing  operational 
analysis  techniques  currently  used  in  capability  development  activities. 
To  meet  their  responsibilities  in  the  acquisition  process,  capability  devel¬ 
opers  must  make  capability  trade-off  decisions  based  on  the  performance 
of  industry  in  meeting  the  requirements,  cost,  and  risk.  The  involvement 
of  industry  prototypes  at  significant  investment  make  close  PM  and 
capability  developer  collaboration  essential  to  understanding  the  TD 
phase  results  and  translating  that  knowledge  into  decisions  that  guide 
the  new  capability  documentation. 

As  draft  requirements  are  provided  to  industry  to  begin  design, 
the  capability  developer  must  remain  actively  engaged  in  the  design 
reviews  for  informed  trade-off  decisions.  Exercising  their  leadership 
in  establishing  the  foundational  requirements,  the  capability  developer 
must  remain  active  in  framing  and  observing  the  results  of  early  key 
testing  to  make  informed  judgments  about  industry’s  success  in  meeting 
the  requirements. 

As  design,  fabrication,  and  test  takes  place,  the  operational  rel¬ 
evance,  feasibility,  and  cost  of  some  requirements  will  be  clear,  but  the 
best  combination  will  not.  Because  the  design  of  a  system  includes  a 
series  of  trade-offs,  indicators  and  issues  on  the  more  critical  decisions  of 
best  balance  in  cost  versus  performance  will  not  be  clear-cut.  The  indica¬ 
tors  will  manifest  themselves  piecemeal  at  various  points  in  design  and 
test,  and  the  capability  developers  use  the  systems  engineering  frame¬ 
work  to  orient  and  correctly  place  indicators  in  a  logical  decision  series 
leading  to  a  sound  capability  statement.  Informed  capability  decision 
making  requires  understanding  the  basics  of  technical  issues  and  using 
that  understanding  to  supplement  user  expertise  to  state  a  feasible  capa- 
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bility.  The  capability  developers  own  the  requirement,  but  the  results  of 
a  competitive  prototyping  TD  phase  will,  by  definition,  produce  changes 
to  the  draft  Capability  Development  Document  (CDD)  used  at  the  start 
of  the  TD  phase.  To  truly  'own'  the  CDD,  capability  developers  need  to 
be  conversant  in  the  basics  of  the  technical  issues  uncovered  in  the  TD 
phase  to  resolve  and  state  the  best  expression  of  feasible  and  useful 
capabilities  in  the  draft  CDD.  Gaining  a  working  understanding  of  the 
technical  issues  involved  in  capability  decisions  will  require  access  to 
technical  resources,  discussed  later  in  this  article. 

A  'Knowledge  Point'- Based  Approach 

Competitive  prototyping  provides  an  immense  array  of  valuable 
information  based  on  the  success  of  the  competing  industry  teams  in 
meeting  performance,  schedule,  and  cost  as  outlined  in  the  TD  phase 
initiating  requirements.  The  primary  goal  of  the  capability  developer 
during  this  phase  is  to  translate  knowledge  gained  in  the  TD  phase 
into  a  technically  achievable,  operationally  relevant,  and  affordable  set 
of  required  capabilities  documented  in  a  revised  CDD.  Abstractly,  the 
capability  developers  could  revise  the  CDD  using  knowledge  of  the  TD 
phase  in  one  of  two  ways:  incrementally,  or  with  a  'big  bang'  at  the  end. 
The  big-bang  approach  presumes  an  extremely  high  level  of  ability  in 
translating  all  of  this  information  and  getting  it  right  in  a  single  change. 
Alternatively,  the  capability  developers  can  play  an  active  role  in  TD 
activities,  incrementally  updating  the  CDD  at  pre-planned  intervals, 
based  on  major  events  in  the  TD  phase  where  key  information  elements 
are  expected  to  be  available.  Incrementally  is  preferred  for  a  number 
of  reasons.  First,  comprehensively  capturing  all  necessary  changes  is 
difficult  over  the  course  of  the  TD  phase:  organizations  often  lose  focus. 
Second,  the  more  revisions  done  at  a  single  point,  the  more  difficult  it  is 
to  manage.  The  more  potential  changes  that  occur  simultaneously,  the 
greater  the  need  for  analysis  resources,  which  can  be  more  efficiently 
used  over  time.  Finally,  an  incremental  approach  allows  capability 
developers  to  identify  an  issue,  establish  an  analysis  team,  conduct  the 
analysis,  and  reflect  the  recommendation  in  a  rigorous  manner. 

We  next  introduce  a  new  term,  Knowledge  Point  (KP),  as  an  approach 
to  address  these  issues.  A  KP  is  a  pre-determined,  event-based  CDD 
review  where  accumulated  knowledge  is  injected  into  the  CDD,  updat¬ 
ing  the  requirements  based  on  analysis  or  test  results .  The  main  idea  is 
to  translate  information  gained  at  key  points  during  the  TD  phase  into 
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actionable  knowledge  to  refine  the  CDD  and  system  specification.  The 
incremental  approach  is  event-driven  and  tied  to  targeted  information 
gaps.  For  a  major  program,  the  capability  developer  may  conduct  four  to 
eight  KPs,  depending  on  the  depth  and  complexity  of  the  initiative  and 
the  length  of  the  TD  phase  activities.  The  number  of  KPs  will  be  driven 
by  the  number  of  key  events  triggering  a  KP  and  the  amount  of  time  avail¬ 
able.  As  time  decreases,  fewer  KPs  may  be  practicable  or  multiple  key 
events  may  be  combined  into  a  single  KP.  Events  that  trigger  a  KP  review 
include:  industry  design  reviews  (Preliminary  Design  Review,  Critical 
Design  Review,  etc.);  the  conclusion  of  major  test  phases  (ballistic  hull 
testing,  performance  testing,  etc.);  and  the  conclusion  of  major  analysis 
activities  (Analysis  of  Alternatives  [AoA],  Trade  Studies,  etc.).  Figure  1 
displays  this  sequence  of  events.  KPs  are  capability  decision  briefs  that 
assess  the  information  available  to  revise  the  CDD.  The  major  result  of 
each  KP  is  a  revised  CDD  with  associated  analysis  products  supporting 
the  decisions  made  at  that  KP.  A  secondary  result  of  KPs  is  to  initiate 
analysis  activities  to  address  the  problems  raised  at  a  particular  KP. 
Such  analyses  and  trade  studies  are  then  due  at  a  future  KP  for  imple¬ 
mentation  in  the  CDD.  To  reduce  confusion  and  ensure  transparency, 
the  capability  developers  only  update  the  CDD  at  KPs,  not  in  between. 

In  planning  a  KP  approach,  the  capability  developers  should  identify 
and  carefully  consider  key  knowledge  gaps  associated  with  the  initiative. 
Which  key  requirements  are  considered  high  risk?  What  are  the  system 
boundaries?  When  are  cost  projections  and  affordability  estimates 
available?  The  program  manager  has  a  responsibility  to  assist  the  capa¬ 
bility  developer  in  identifying  these  knowledge  gaps.  In  a  well-designed 
program,  information  about  these  knowledge  gaps  will  be  addressed  by 
the  TD  phase  events  planned  by  the  program  manager.  For  example,  fea¬ 
sible  protection  requirements  are  addressed  in  live  fire  testing;  feasible 
reliability  is  assessed  in  durability  testing;  weight  is  assessed  in  design 
reviews  and  upon  prototype  arrival  at  test  centers.  Where  knowledge 
gaps  are  not  addressed,  the  capability  developers  must  work  with  the  pro¬ 
gram  manager  to  get  these  key  knowledge  gaps  addressed  in  the  planned 
activities.  The  capability  developers  must  also  consider  when  this  infor¬ 
mation  is  available  with  respect  to  the  CDD  development  timeline,  and 
work  with  the  testing  and  cost  authorities  to  ensure  that  their  products 
are  available  early  enough  to  influence  the  CDD  refinement  activities. 
Stove-piped  delivery  of  test  results  and  cost  estimates  that  are  not  avail- 
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Notes.  SRR  =  System  Requirements  Review;  PDR  =  Preliminary  Design  Review;  CDR  =  Critical  Design  Review;  TRR  =  Test  Readiness  Review. 
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able  until  very  late  in  the  TD  phase  will  not  support  the  CDD  decision 
timeline.  Collaboration  is  required  to  sequence  test  activities  and  cost 
analysis  activities  to  address  key  concerns  early  using  interim  reports. 

Implementing  a  KP-based  approach  to  incrementally  refining  the 
CDD  provides  several  key  benefits.  First,  it  provides  a  framework  upon 
which  PMs  can  base  their  own  plans,  synchronizing  the  overall  effort. 
Specification  development  activities  can  base  their  development  plans 
from  the  KP  timeline.  The  AoA  and  cost  analysis  teams  can  use  specific 
KPs  as  a  data  cut-off  point.  Key  tests  can  be  scheduled  to  ensure  results 
are  available  to  inform  the  CDD.  Importantly,  this  approach  ensures 
transparency  in  how  analysis  and  test  results  are  used  to  drive  key  CDD 
decisions.  Second,  all  CDD  decisions  are  implemented  in  an  open  KP  for¬ 
mat  with  key  stakeholders  present.  Transparency  eliminates  confusion, 
allowing  sequential  decisions  by  the  systems  engineering,  test,  or  cost 
organizations  to  proceed  with  the  best  information  about  the  intent  of  the 
decision  and  the  constraints  under  which  it  was  made.  Third,  a  knowl¬ 
edge  point,  incremental  approach  allows  for  the  full  impact  of  a  decision 
to  be  clarified  or  revisited  as  the  phase  progresses.  To  summarize,  having 
a  series  of  KPs  supports  a  deliberate  analytical  process  in  which  issues 
are  sequentially  identified  and  framed  with  assumptions,  analyses  are 
conducted,  and  recommended  solutions  are  then  presented  to  leadership 
for  decisions  and  recorded  in  the  newest  CDD  draft. 

Executing  A  Typical  Knowledge  Point 

The  capability  developers  must  own  the  KP  process.  Each  KP  event 
should  be  structured  as  a  decision  brief  with  defined  decision  authority. 
Decision  authority  is  discussed  in  more  detail  later  in  this  article.  The 
Requirements  Integrated  Product  Team  leading  up  to  each  KP  is  the  place 
for  detailed  discussions  and  development  of  recommended  positions  on 
each  issue,  allowing  the  KP  to  be  focused  on  the  'so  what’  of  key  analysis 
or  test  results.  The  agenda  of  each  KP  can  include  updates  of  ongoing 
studies,  but  is  effective  when  focused  on  a  final  results  briefing  of  com¬ 
pleted  analyses  ready  for  decision.  While  large  groups  tend  to  complicate 
decision  making,  KP  attendees  should  include  all  the  key  stakeholders  for 
stable  decisions,  ensuring  transparency.  Two  stakeholders,  the  PM  and 
lead  systems  engineer,  hold  special  prominence  at  KP  reviews  as  they  hold 
the  most  accurate  assessments  of  technical  feasibility,  maturity,  and  cost 
and  schedule  risk.  The  capability  developer  plans,  coordinates,  and  leads 
the  analysis  activities,  often  relying  on  the  PM  or  other  technical  expert 
to  assist  in  the  conduct  of  each  analysis  activity.  When  analysis  and  test 
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activities  are  ready  for  presentation,  the  capability  developer  and  techni¬ 
cal  experts  collaborate  in  presenting  the  material  at  the  KP.  Later  in  this 
article,  we  discuss  access  to  technical  resources,  which  is  a  key  enabler 
of  sound  capability  development  decisions. 

Each  KP  should  include  success  criteria  to  assist  in  communicat¬ 
ing  with  stakeholders  the  focus  of  each  KP  event.  The  results  of  the  KP 
should  be  summarized  in  a  Memorandum  for  Record  (MFR)  stored  in 
a  location  accessible  to  those  who  need  to  reference  it.  The  capability 
developers  and  PMs  will  reorganize  their  efforts  after  each  KP  to  ensure 
they  are  correctly  aligned  with  the  overall  direction  of  the  capability 
development  effort  based  on  the  decisions  made  at  the  KP.  Therefore,  the 
KP  and  its  results  must  be  accessible.  External  agencies  will  also  seek 
to  minimize  disruptions  to  a  program,  and  can  use  KPs  as  key  interface 
points  with  which  to  engage  a  program. 

Well  in  advance  of  each  KP,  the  capability  developers  provide  a 
draft  copy  of  the  CDD  with  which  stakeholders  are  invited  to  generate 
comments.  Using  a  standardized  format,  such  as  the  existing  JROC 
Knowledge  Management  Decision  Support  Comment  Resolution  Matrix 
is  recommended  for  simplicity.  Comments  from  the  stakeholders  should 
be  returned  with  sufficient  time  (approximately  2  weeks)  prior  to  the 
KP  event  to  allow  time  for  background  work  to  be  conducted  on  each 
comment.  The  capability  development  team  takes  each  change  rec¬ 
ommendation  and  conducts  an  impact  (traceability)  assessment  to 
determine  which  related  CDD  attributes  would  be  affected  by  the  change. 
Each  comment  is  characterized  as  a  ‘non-issue,’  ‘major  analysis,’  ‘minor 
analysis,’  or  a  ‘deferral.’  Changes  that  didn’t  require  analysis  (i.e.,  could 
be  accepted  or  rejected  without  further  effort)  are  characterized  as 
‘non-issues.’  Changes  where  insufficient  data  exist  or  where  the  answer 
will  be  available  at  a  defined  future  event  (such  as  a  test)  are  character¬ 
ized  as  'deferral'  and  are  deferred  until  the  correct  data  are  available. 
Changes  proposed  to  critical  requirements  or  requiring  further  analysis 
are  characterized  as  major  analyses.  Changes  requiring  further  analysis 
and  proposed  to  lower  tiered,  non-KPP  requirements  are  characterized 
as  minor  analyses.  Once  comments  requiring  analysis  are  characterized 
the  study  objectives  are  determined,  and  guidance  and  resources  are 
assigned  (Figure  2). 
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FIGURE  2.  EXECUTING  THE  KNOWLEDGE  POINT  MACRO  PROCESS 


Once  the  background  work  is  completed,  the  results  are  published  3 
days  prior  as  a  read-ahead  for  the  KP.  This  allows  participants  to  arrive 
knowing  all  of  the  salient  issues  and  understanding  the  decisions  needed 
at  the  KP.  The  background  work  reflects  the  recommended  ‘going-in’ 
positions  at  the  KP.  However,  no  decisions  are  made  except  at  the  KP  to 
ensure  transparency.  At  the  KP,  each  proposed  change  with  supporting 
analysis  is  reviewed,  and  the  CDD  decision  authority  adjudicates  the  pro¬ 
posed  changes  after  receiving  input  from  key  stakeholders.  Those  changes 
adjudicated  as  major  analyses,  minor  analyses,  or  deferrals  are  tagged  as 
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‘on-hold’  and  tracked  in  the  requirements  management  database.  Deci¬ 
sions  on  each  proposed  change  are  made  only  at  KPs  when  the  analysis 
is  complete,  not  necessarily  when  the  proposed  change  is  first  submitted. 

KPs  should  include  the  use  of  metrics  and  culminate  in  a  decision 
to  either  publish  a  revised  CDD  or  publish  an  erratum.  This  provides  a 
quantitative  snapshot  regarding  requirements  uncertainty,  detailing 
what  studies  have  been  closed  and  implemented,  as  well  as  which  are 
outstanding.  It  reflects  how  many  change  proposals  are  being  submitted 
at  a  given  time  and  helps  assess  relative  success  at  dealing  effectively 
with  the  complete  set  of  proposed  changes. 

Key  to  sound  decision  making  is  the  rigorous  use  of  analysis  and  test 
results  to  underpin  every  activity  and  decision.  Deferring  decisions  until 
sufficient  information  is  available  is  preferable  to  changing  an  attribute 
or  CDD  section  multiple  times.  Reliance  on  test  results  and  technical 
analyses  moderates  the  influence  of  any  one  stakeholder  group.  While  not 
always  easy,  making  CDD  decisions  only  at  the  KP  is  key  to  maintaining 
transparency  and  critical  to  reinforcing  the  goal  of  always  underpinning 
every  CDD  change  based  on  analysis  or  test  results.  The  results  of  each 
KP  should  be  communicated  throughout  the  capability  developer  and  PM 
organizations  to  ensure  everyone  understands  how  these  decisions  affect 
their  own  work.  This  can  be  accomplished  via  an  MFR  summarizing 
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the  KP  outcomes.  Publishing  an  MFR  ensures  only  one  (vice  multiple) 
interpretations  of  a  decision  made,  which  is  especially  important  if  the 
KP  decision  is  to  publish  an  errata,  rather  than  an  updated  draft  CDD. 

For  key  requirements,  the  decision  authority  for  a  CDD  change  is  typ¬ 
ically  the  capability  development  senior  leadership.  Therefore,  following 
select  KPs  with  a  General  Officer— or  SES-level  senior  leadership  review 
—is  useful  to  validate  key  decisions.  For  example,  a  senior  leader  review 
can  be  used  to  validate  the  Key  Performance  Parameter  (KPP)  or  to  vali¬ 
date  a  key  trade-off  decision  with  far-reaching  effects.  This  ensures  that 
the  Service  leadership  remains  engaged  in  the  capability  development 
initiative,  and  can  serve  as  a  forum  to  reconcile  differences  that  could 
not  be  resolved  at  the  action  officer  level.  However,  to  preclude  schedule 
slip,  these  reviews  should  be  scheduled  in  advance.  Additionally,  the 
scheduling  of  senior  leader  reviews  should  balance  their  availability  and 
authority  with  the  substance  of  the  issues  being  reviewed. 

Early  Use  of  Systems  Engineering  Fundamentals 

Early  use  of  systems  engineering  fundamentals  is  essential  to  suc¬ 
cessfully  implementing  a  KP-based  capability  development  approach. 
Key  tenets  include:  (a)  determine  the  plan  upfront;  (b)  application  of  best 
practices;  (c)  enterprise-level  use  of  requirements  management  software; 
(d)  access  to  technical  resources;  (e)  integrating  test  results;  and  (f)  early 
and  ongoing  cost  integration. 

A  comprehensive  technical  plan  is  essential  during  the  TD  phase:  our 
warfighters  depend  on  us,  and  a  significant  amount  of  taxpayer  money 
is  involved  in  any  TD  phase  initiative.  The  plan  should  address  the  tim¬ 
ing,  events,  and  execution  of  various  KPs  and  the  knowledge  gaps  they 
seek  to  resolve.  It  should  address  what  roles  and  responsibilities  various 
organizations  will  play  in  terms  of  issue  identification,  analysis,  decision 
authority,  and  closure.  Given  that  potentially  a  lot  of  changes  to  a  CDD 
and  systems  specification  can  occur,  how  will  these  changes  be  tracked, 
managed,  and  burned  down?  How  will  analyses  initiated  at  a  given  KP 
be  tracked  and  managed?  Decision  authority  is  especially  important.  At 
each  KP,  the  lead  capability  developers  should  make  CDD -relevant  deci¬ 
sions  after  hearing  the  key  points  of  stakeholders,  with  special  attention 
paid  to  the  PM  and  lead  systems  engineer.  Certain  key  decisions,  such 
as  regarding  a  KPP,  should  be  validated  following  select  KPs  at  a  senior 
leader  review.  All  of  these  decisions  are  re-validated  as  the  CDD  moves 
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through  Service  and  Joint  staffing  as  well  as  during  key  acquisition 
meetings  such  as  the  Defense  Advisory  Boards.  Finally,  the  plan  should 
address  how  software  will  be  used  in  the  process  for  key  activities  like 
requirements  management,  test  integration,  and  include  how  any  classi¬ 
fied  aspects  will  be  handled.  While  many  of  these  are  simple,  they  must 
be  documented  to  ensure  common  understanding  given  the  number  of 
people  involved  in  a  large  program.  Some  decisions  are  not  at  all  simple 
and  require  forethought  and  planning.  All  of  these  decisions  and  the 
resulting  plan  should  be  documented  in  the  Requirements  Management 
and  Analysis  Plan  (RMAP)  and  signed  by  each  of  the  lead  capability 
developers  and  PMs.  Implementing  this  plan,  including  the  sections 
described  below,  requires  an  investment  of  resources  by  the  capabil¬ 
ity  developer  in  terms  of  people  and  funding,  and  a  commitment  to  the 
processes  it  describes.  For  the  capability  developer,  this  may  require  one 
to  three  additional  staff  members  to  execute  this  process,  depending  on 
the  status  of  the  program.  No  additional  staff  is  needed  for  the  PM.  To 
keep  the  RMAP  from  growing  stale,  it  can  be  reviewed  at  each  KP  to 
determine  if  changes  should  be  made  in  the  plan.  A  copy  of  the  techni¬ 
cal  plan  used  to  execute  the  KP  process  on  JTLV  (Pflanz  &  Clark,  2009) 
is  available  through  the  Defense  Technical  Information  Center  (DTIC) 
Online  Access  Controlled  as  accession  SURVIAC-SV-33264. 

Best  practices  in  systems  engineering,  as  taught  at  the  Defense 
Acquisition  University,  must  continue  to  make  their  way  into  capa¬ 
bility  development  activities.  This  principle  aligns  with  the  general 
guidance  of  the  JCIDS  as  described  in  Chairman  Joint  Chiefs  of  Staff 
Instruction  3170.  However,  certain  aspects  are  particularly  important 
and  worth  elaboration.  First,  the  attributes  in  the  CDD  should  include 
decomposition  and  relative  prioritization  (Figure  3).  Decomposition 
is  important  because  it  describes  how  a  top-level  capability,  such  as  a 
KPP,  is  supported  by  lower  level  capabilities.  A  functional  hierarchy 
can  be  developed  to  support  decomposition  using  existing  systems  engi¬ 
neering  techniques.  When  doing  the  impact  assessment  during  the  KP 
process,  this  decomposition  can  be  used  to  support  the  impact  (trace- 
ability)  analysis  to  determine  what  other  requirements  are  affected  by 
a  single  attribute  change.  Relative  prioritization  is  equally  important. 
It  can  be  used  to  inform  trade-off  decisions  during  the  KP  process  to 
preclude  lower  level  attributes  from  causing  undue  performance  or 
cost  risk  to  high-priority  capability,  such  as  a  KPP.  Relative  priority 
also  can  be  flowed  down  into  the  system  specification.  Relative  priority 
can  be  established  by  assessing  an  attribute’s  ‘depth’  in  the  functional 
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hierarchy  and  through  subject  matter  expertise.  Relative  priority  can  be 
reflected  using  a  set  of  tiers,  where  the  definition  for  each  tier  is  clearly 
defined  (Figure  3).  It  is  essential  that  industry  understand  the  relative 
priority  of  the  requirements  for  it  to  make  sensible  trade-off  decisions 
when  building  prototypes.  While  the  CDD  is  important,  the  warfighter 
ends  up  receiving  what  industry  builds,  and  industry  will  build  to  the 
system  specification,  not  the  CDD.  Therefore,  conducting  a  series  of 
CDDs  to  system  specification  crosswalks  is  absolutely  essential  to  suc¬ 
cess.  These  crosswalks  should  verify  that  each  attribute  is  completely 
and  accurately  decomposed,  and  that  there  are  no  requirements  in  the 
system  specification  without  a  parent  in  the  CDD.  The  results  of  these 
series  of  crosswalks  should  be  agreed  to  by  senior  leadership  at  a  formal 
review  prior  to  strategic  points  in  the  acquisition  process.  This  is  critical 
to  ensuring  the  system  specification  is  a  sufficient  and  accurate  repre¬ 
sentation  of  warfighter  needs  stated  in  the  CDD. 

Enterprise-level  use  of  requirements  management  software  is  a 
key  enabler  to  rigorous  execution  of  the  KP  process.  IBM’s  DOORS© 
is  one  popular  software  package.  The  increased  demands  on  perfor¬ 
mance,  complexity,  and  costs  of  systems  now  being  developed  require 
tight  coupling  between  operational  requirements  stated  in  the  CDD, 
system  requirements  stated  in  the  specification,  and  test  results.  All  of 
the  requirements  documents,  such  as  the  CDD  and  system  specifica¬ 
tion,  need  to  be  resident  in  a  single  database  with  controlled  access  for 
authorized  staff  among  capability  developers,  PMs,  and  testers.  Since 
capability  developers  and  PMs  will  often  be  geographically  separated, 
this  may  require  a  networked  tool  to  allow  all  data  to  reside  on  a  single 
database.  The  more  often  the  CDD  is  updated,  the  more  frequently  all  of 
those  ripple  effects  will  flow  down  toward  related  and  child-level  docu¬ 
ments.  While  it  maybe  physically  possible  to  manage  a  CDD  in  MS  Word, 
doing  so  is  not  recommended.  Changes  will  get  lost  or  misapplied.  The 
program  office  will  have  difficulty  in  tracing  requirements  and  decom¬ 
posing  requirements  as  they  change. 

Access  to  technical  resources  is  also  essential  to  effective  execu¬ 
tion  of  the  KP  process.  A  large-scale  capability  development  effort  will 
include  a  wide  variety  of  technical  aspects.  No  one  organization  or 
individual  can  be  expected  to  provide  technical  expertise  across  the 
spectrum.  Establishing  a  working  relationship  and  ensuring  access 
to  technical  experts  in  the  government  are  essential  to  success.  The 


436 


Defense ARJ,  October  2012,  Vol.  19  No.  4 : 422-442 


Applying  Early  Systems  Engineering:  Injecting  Knowledge  into  the  Capability  Development  Process 


government  has  established  centers  of  excellence  in  almost  every  area 
of  science  and  engineering  relevant  to  weapon  systems  development 
and  should  be  included  where  possible.  By  resourcing  these  agencies  to 
conduct  analyses  to  support  capability  development,  capability  devel¬ 
opers  get  access  to  the  best  minds  in  government  who  are  already  ‘past 
the  learning  curve’  on  the  particular  issue  at  hand.  Importantly,  a  capa¬ 
bility  development  effort  should  establish  a  standing  Whole  Systems 
Trade  Study  (WSTS)  group.  The  WSTS  group  focuses  on  whether  the 
KPPs  and  other  key  requirements  are  achievable  at  the  whole  system 
level.  An  example  of  one  such  group  is  the  U.S.  Army  Tank  Automotive 
Research,  Development  and  Engineering  Center,  Advanced  Concepts 
Lab  (TARDEC  ACL).  On  the  JLT  V  program,  the  TARDEC  ACL  served  as 
a  WSTS  Group  by  building  full  computer  models  of  a  government  design 
for  JLTV,  and  also  analyzing  industry  designs  as  they  matured.  They 
analyzed  whole  system  achievability,  as  well  as  manipulating  designs  to 
answer  ‘what  if  questions.  The  WSTS  group  government  designs  were 
also  used  as  alternatives  in  the  AoA,  and  portions  of  the  WSTS  group 
participated  in  the  AoA.  For  JLTV,  the  WSTS  Group  was  especially 
important  in  the  decision  to  increase  the  JLTV  underbody  protection 
requirements  and  determine  which  other  system  requirements  must  be 
traded.  Here,  the  computer  models  proved  invaluable  to  underpinning 
this  key  protection  decision. 

Integrating  test  results  is  a  key  enabler  of  effectively  executing  the 
KP  process.  The  test  results  must  show  that  the  current  requirements 
stated  in  a  CDD  for  a  program  at  Milestone  B  are  achievable;  or  where 
modified  from  the  delivered  prototypes,  they  are  estimated  as  achievable 
by  a  credible  expert  authority  or  analytical  modeling  result.  Assessing 
test  results  is  difficult  because  it  involves  a  complex  mapping  of  multiple 
prototype  test  results  used  to  assess  achievability  and  the  fact  that 
requirements  often  changed  during  execution  of  the  KPs.  Collaboration 
of  the  testers,  systems  engineers,  and  capability  developers  is  required 
to  sufficiently  translate  the  test  information  into  actionable  knowledge 
that  can  be  applied  in  the  CDD. 

Test  results  are  traditionally  available  at  the  end  of  testing,  and 
therefore  the  end  of  the  TD  phase.  This  is  not  compatible  with  a  com¬ 
petitive  prototyping  TD  phase  where  the  requirements  are  periodically 
updated  as  described  previously.  However,  a  prioritized  test  schedule 
can  be  developed  using  phases  where  test  results  are  available  at  the 
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end  of  each  phase.  KPs  can  be  tied  to  the  timing  of  each  test  phase.  If 
done  in  priority  order,  the  most  important  CDD  attributes  are  verified 
first,  with  lower  importance  attributes  varied  as  the  testing  progresses. 
There  will  be  important  exceptions  to  this  rule.  For  example,  durability 
testing  typically  involves  long  durations;  therefore,  reliability  and  cer¬ 
tain  sustainment  attributes  cannot  be  verified  until  late  in  the  phase. 
However,  these  exceptions  can  be  dealt  with  while  still  verifying  as  many 
key  attributes  as  early  as  possible. 

The  purpose  of  the  TD  phase  is  to  “get  the  requirements  ‘right’”; 
therefore,  a  logical  consequence  of  the  TD  phase  is  changed  require¬ 
ments.  Where  possible,  the  test  plans  should  be  modified  to  reflect  new 
changes  to  the  CDD  at  a  prior  KP.  For  example,  if  a  KPP  changes  or 
the  mission  profile  changes  in  time  to  be  reflected  in  testing,  then  the 
program  will  benefit  from  testing  to  the  new  requirement  vice  the  old 
requirement.  Not  passing  a  modified  requirement  (to  which  industry  did 
not  design  toward)  does  not  necessarily  invalidate  the  new  requirement; 
however,  it  does  increase  the  level  of  uncertainty  in  the  achievability 
of  that  attribute.  Finally,  it  is  essential  for  capability  developers  to  be 
present  at  certain  key  test  events  to  collect  the  ‘right’  take-aways  from 
the  test  and  to  ensure  that  the  testing  is  in  accordance  with  the  implicit 
vision  and  explicit  attributes  of  the  CDD. 
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Early  integration  of  cost  estimates  is  the  last  key  enabler  to  the  KP 
process.  Test  results  and  delivered  prototypes  may  demonstrate  achiev- 
ability,  but  not  affordability.  Correlating  cost  estimates  from  prototype 
work  to  affordability  estimates  requires  early  integration  of  the  cost 
estimating  profession.  This  requires  several  key  activities.  First  is  estab¬ 
lishing  a  cost  threshold  beyond  which  the  system  is  at  risk  of  not  being 
affordable;  in  the  case  of  JLTV,  this  meant  establishing  cost  as  a  Key 
System  Attribute.  Second  is  to  correlate  cost-driving  requirements  with 
the  relative  priority  of  requirements.  Using  a  cost-informed  trade-off 
assessment,  the  capability  developer  must  be  prepared  to  make  difficult 
trade-off  decisions  to  ensure  low-priority,  cost-driving  requirements 
do  not  price  the  capability  above  the  affordability  cutline.  Typically, 
these  cost  trade-off  decisions  require  the  participation  of  senior  leaders. 
This  integration  of  cost  analysis  is  similar  in  scope  to  the  DoD’s  Better 
Buying  Power  Initiative  (https://dap.dau.mil/bbp)  where  cost  analyses 
are  used  to  inform  systems  engineering  trade-off  decisions  to  meet  an 
affordability  target. 


Conclusions 

This  article  described  a  new  approach  to  capability  development  in 
the  DoD’s  new  competitive  prototyping  guidance  for  the  TD  phase.  It 
focused  on  how  the  draft  requirement  is  refined  through  a  series  of  KPs, 
enabled  by  early  use  of  systems  engineering  fundamentals.  By  follow¬ 
ing  the  ideas  established  in  this  approach,  future  programs  can  tailor 
their  application  based  on  program  peculiarities;  however,  the  common 
principles  described  here  will  endure  regardless  of  scope  or  application. 
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Improving  Acquisition 
Outcomes  Through 
Simple  System  Technology 
Readiness  Metrics 


Chad  L.  Dacus 


This  article  advocates  the  use  of  simple  technology  readi¬ 
ness  metrics  that  focus  on  system-wide  technological 
maturity.  Current  DoD  practice  is  to  set  guidelines  for 
the  maturity  of  individual  system  components,  but  the 
statistical  evidence  provided  in  this  article  demonstrates 
that  more  holistic  metrics  should  be  adopted.  A  simple 
system  technology  readiness  metric  is  proposed  and 
evaluated  based  on  historical  cost  and  schedule  perfor¬ 
mance,  and  is  shown  to  be  potentially  quite  useful  in 
avoiding  poor  acquisition  outcomes.  Finally,  the  policy 
implications  of  implementing  a  decision  rule  based 
on  the  metric  are  explored  in  depth,  and  the  DoD  is 
advised  to  pursue  and  encourage  applied  research  for 
the  development  of  more  comprehensive  technology 
readiness  metrics. 


444 


Defense  ARJ,  October  2012,  Vol.  19  No.  4 : 444-461 


image  designed  by  Diane  Fleischer  » 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


The  discovery  process  in  defense  acquisition  is  expensive  and  time- 
consuming.  If  the  Department  of  Defense  (DoD)  is  too  optimistic  in  what 
technologies  it  believes  U.S.  defense  contractors  can  master  in  a  timely 
fashion,  taxpayers  will  be  subject  to  large  cost  overruns,  and  warfighters 
will  go  without  more  effective  weaponry  for  much  longer  than  expected. 
Congress  has  attempted  to  reduce  this  risk  by  codifying  minimum  tech¬ 
nology  maturity  levels  for  program  elements  before  they  can  be  included 
in  a  program  of  record.  To  further  reduce  these  risks,  minimum  technol¬ 
ogy  maturity  levels  should  be  enforced  for  the  program  in  its  entirety 
rather  than  focusing  exclusively  on  its  individual  components.  Indeed, 
empirical  evidence  gleaned  from  an  evaluation  of  the  effect  of  Technol¬ 
ogy  Readiness  Levels  (TRL)  on  system  cost  overruns  and  schedule  slips 
strongly  supports  taking  a  more  holistic  view  of  technology  maturity. 
To  put  the  magnitude  of  possible  savings  in  perspective,  a  potentially 
avoidable  additional  cost  overrun  of  40  percent  through  the  procurement 
phase  for  a  single  ‘typical’  $2.5  million  Major  Defense  Acquisition  Pro¬ 
gram  (MDAP)  results  in  $1  billion  in  additional  outlays.  The  DoD  should 
stop  reaching  for  elusive  technologies  in  its  programs  of  record,  but  it 
should  continue  to  retain  and  even  enhance  its  technology  leader  status 
through  strengthening  its  science  and  technology  programs. 

The  Services  face  a  problem  familiar  to  commercial  interests  in 
high-technology  sectors:  the  development  and  fielding  of  appropriate 
technologies  to  satisfy  customers’  demands.  The  connection  of  tech¬ 
nology  to  system  requirements  is  clear— higher  levels  of  technology 
generally  allow  the  satisfaction  of  more  demanding  requirements.  How¬ 
ever,  more  advanced  technologies  can  take  much  more  time  and  money  to 
develop,  so  trade-offs  have  to  be  made  in  a  resource-constrained  world. 

TRLs  describe  the  state  of  a  critical  technology  element’s  develop¬ 
ment.  The  National  Aeronautics  and  Space  Administration  developed 
the  TRL  methodology  in  1974  (Banke,  2011).  The  DoD  adopted  the  TRL 
framework,  and  it  is  now  partially  codified  in  regulations  applying  to  all 
MDAPs.  Title  10  United  States  Code  (U.S.C.)  Section  2366b  requires  the 
Milestone  Decision  Authority  to  verify  that  “the  technology  in  the  [major 
defense  acquisition]  program  has  been  demonstrated  in  a  relevant  envi¬ 
ronment”  before  receiving  Milestone  B  approval.  A  technology  readiness 
assessment  consists  of  classifying  each  critical  technology  element  into 
one  of  nine  technology  readiness  categories  according  to  its  technologi¬ 
cal  maturity.  The  DoD  uses  the  following  scale  (DoD,  2009): 
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•  TRL  1:  Basic  principles  observed  and  reported 

•  TRL  2:  Technology  concept  and/or  application  formulated 

•  TRL  3:  Analytical  and  experimental  proof  of  concept 

•  TRL  4:  Component  validation  in  a  laboratory 

•  TRL  5:  Component  validation  in  a  relevant  environment 

•  TRL  6:  Subsystem  model  or  prototype  demonstration  in  a 
relevant  environment 

•  TRL  7:  System  prototype  demonstration  in  an  operational 
environment 

•  TRL  8:  Actual  system  completed  and  qualified  through  test 
and  demonstration 

•  TRL  9:  Actual  system  proven  through  successful  mission 
operations 

As  is  clear  from  Title  10’s  language,  system  components  must  achieve 
TRL  6  before  reaching  Milestone  B.  Although  not  codified  in  the  U.S.C., 
TRL  7  or  higher  is  the  expected  state  of  technology  maturity  at  Milestone 
C.  In  addition,  some  programs  use  technology  readiness  assessments 
as  an  integral  part  of  their  risk  assessment  and  risk  reduction  strategy 
(DoD,  2009).  Since  production  begins  after  the  Milestone  C  decision, 
a  strong  argument  can  be  made  that  testing  and  integration  should  be 
complete— rendering  TRL  8  the  more  appropriate  standard. 

The  technological  maturity  of  a  system’s  critical  components  has 
long  been  recognized  as  a  key  determinant  of  weapon  systems  outcomes 
(General  Accounting  Office,  1999).  If  significant  technological  advances 
are  required  during  design  and  manufacturing  development,  the  pro¬ 
gram  will  be  very  susceptible  to  extended  cycle  times,  higher  unit  costs, 
management  changes,  and  funding  volatility.  In  addition,  these  outcomes 
will  provoke  deleterious  second-order  effects  such  as  smaller  buys  and 
technological  obsolescence  of  more  mature  system  components.  Tech¬ 
nological  obsolescence  can  then  lead  to  requirements  creep.  It  is  easy 
to  see  how  a  vicious  cycle  could  take  hold  and  even  lead  to  a  program’s 
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cancellation.  Expanding  technology  readiness  metrics  by  including  a 
simple  indicator  of  system  readiness  will  allow  program  leadership  to 
effectively  control  risk  at  both  the  individual  technology  element  level 
and  at  the  system  level. 

As  mentioned  previously,  the  U.S.C.  requires  an  overall  minimum 
TRL  of  6  for  critical  technology  elements.  The  Government  Account¬ 
ability  Office  (GAO)  has  recommended  that  technologies  included  in  a 
product’s  design  reach  TRL  7  before  being  turned  over  to  the  product 
development  manager  (GAO,  2009).  In  addition  to  reinforcing  the  GAO’s 
position,  empirical  analyses  of  Selected  Acquisition  Reports  (SAR)  and  a 
system’s  TRLs  support  including  an  additional  metric  that  takes  a  more 
comprehensive  view  of  technological  maturity. 

Research  Methods 

The  quantities  of  interest  for  this  analysis  are  the  percentage  change 
from  the  earliest  available  estimated  Program  Acquisition  Unit  Cost 
(PAUC)  to  the  current  estimated  PAUC  and  the  actual  schedule  slippage, 
in  months,  as  of  the  last  published  SAR.  TRLs  were  taken  from  Defense 
Acquisition  Executive  Risk  Summaries  when  these  numbers  became 
available  starting  in  March  2007.  All  systems  included  in  the  sample 
had  reached  Milestone  B  by  the  time  TRLs  were  collected.  To  exploit 
recently  available  technology  readiness  data,  programs  were  selected  for 
inclusion  in  the  sample  if  Milestone  B  approval  occurred  after  January 
1, 2000.  Programs  reaching  Milestone  B  before  2000  were  very  likely  to 
have  TRLs  of  8  or  better  for  every  critical  technology  element  by  the  time 
TRL  data  became  available  in  2007,  so  the  additional  data  would  not  be 
particularly  informative. 

While  a  minimum  TRL  is  prescribed  by  law  at  Milestone  B,  the  raw 
number  of  critical  technology  elements  with  TRLs  below  8  has  not  been 
identified  as  a  major  risk  factor.  For  the  purposes  of  this  analysis,  when 
a  critical  technology  element’s  TRL  reaches  8,  technology  risk  has  been 
effectively  eliminated.  To  keep  the  focus  on  shortfalls  in  technology  and 
because  of  mathematical  necessity,  the  candidate  explanatory  variables 
were  stated  with  the  difference  between  a  critical  technology  element’s 
TRL  and  a  TRL  of  8  as  the  basic  building  block.  In  mathematical  nota¬ 
tion,  this  shortfall  can  be  found  quite  simply  by  computing: 

i-th  critical  technology  element’s  TRL  shortfall  =  8  -  TRL 
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For  example,  the  minimum  TRL  of  6  that  is  codified  in  the  U.S.C.  can 
be  written  interchangeably  as 

min  TRL  =6  or 

X 

max(8  -  TRL))= 2 

The  same  principle  can  be  applied  for  all  critical  technology  elements 
with  TRLs  less  than  or  equal  to  8.  Armed  with  the  insight  that  both 
the  number  and  magnitude  of  technology  shortfalls  are  important,  the 
candidate  explanatory  variables  containing  TRL  information  that  were 
considered  are  listed  in  Table  1. 


TABLE  1.  CANDIDATE  EXPLANATORY  VARIABLES 


Quantity  Measured 

Mathematical  Specification 

Number  of  Technology  Issues 

N  =  Count(8  -  TRLj )  >  0 

Number  and  Severity  of  Issues 

N 

*  ^ 

Sum  =  /.  ( 8  -  TRLj ) 

/  =  1 

Number/Weighted  Severity  of  Issues 

N 

SS  =  ^  (8  -  TRLj)2 
/  =  1 

Maximum  Technology  Shortfall 

Max  =  Max(8  -  TRLj) 

The  sum  of  the  squared  TRL  shortfalls,  or  SS,  merits  additional 
explanation.  Clearly,  advancing  from  a  TRL  of  5  to  a  TRL  of  6  may  not 
require  the  same  amount  of  effort  as  advancing  from  a  TRL  of  6  to  TRL 
7.  That  is,  there  is  no  reason  to  believe  TRLs  were  designed  as  a  linear 
scale.  Squaring  the  TRL  shortfall  allows  more  serious  shortfalls  to  be 
weighted  much  more  heavily  than  minor  shortfalls.  The  effect  of  squar¬ 
ing  is  pronounced— a  shortfall  of  two  units  in  the  TRL  scale  is  weighted 
four  times  more  heavily  than  a  shortfall  of  one.  A  shortfall  of  three  is  con¬ 
sidered  to  be  a  major  weakness  and  is  given  a  weight  nine  times  higher 
than  a  shortfall  of  one  to  reflect  its  relative  seriousness. 
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These  candidate  variables  and  combinations  of  them  were  regressed 
against  the  number  of  months  the  system  is  or  is  projected  to  be  behind 
Acquisition  Program  Baseline  (APB)  schedule  and  the  ratio  of  current 
estimated  PAUC  to  the  earliest  available  comparable  estimate  of  PAUC. 
To  preserve  the  simplicity  of  the  model  and  to  keep  the  number  of  deci¬ 
sion  rules  to  a  minimum,  a  variable  or  combination  of  variables  was 
evaluated  according  to  its  ability  to  explain  both  schedule  slippages  and 
the  percentage  change  in  estimated  PAUC.  In  addition  to  evaluating  the 
effects  of  various  technology  maturity  metrics,  several  other  explanatory 
variables  were  incorporated  in  models  to  assess  their  relative  value  in 
explaining  acquisition  outcomes.  These  covariates,  and  whether  each 
variable  helped  explain  acquisition  outcomes,  are  listed  in  Table  2. 


TABLE  2.  OTHER  VARIABLES  CONSIDERED  AND  THEIR 
EXPLANATORY  VALUES 


Independent  Variable 

Explanatory  Value 

Service 

No 

Months  Since  Milestone  B 

Yes 

Type  of  Commodity 

No 

Prime  Contractor 

No 

Cost  Variance  Causes 

Yes 

Program  Size 

No 

The  rationale  for  the  consideration  of  most  of  these  covariates  is 
straightforward.  The  lead  Service,  the  type  of  commodity  being  acquired, 
and  the  prime  contractor  were  included  to  determine  whether  acquisition 
outcomes  differed  systematically  because  of  the  personnel  involved  in 
running  the  program  or  the  fundamental  nature  of  the  acquisition  itself. 
None  of  these  were  found  to  be  useful  in  explaining  acquisition  outcomes 
when  any  TRL  variable  was  also  included  in  the  model.  A  program’s  size, 
measured  by  its  estimated  total  acquisition  cost,  was  considered  because 
expenditure  could  be  a  good  proxy  for  complexity,  so  larger  programs  may 
be  intrinsically  more  difficult  to  gauge.  No  statistical  evidence  was  found 
to  support  this  conjecture.  Finally,  sources  of  cost  variance  identified  in 
the  SAR  were  introduced  in  various  models  to  determine  whether  one 
particular  type  of  error  was  particularly  influential  in  explaining  acqui¬ 
sition  outcomes.  Even  though  information  on  these  variables  is  obviously 
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collected  after  the  fact,  they  can  still  shed  light  on  the  consequences  of 
misjudging  a  program  early  in  its  development.  One  of  these  indicator 
variables  was  found  to  be  useful  in  explaining  each  acquisition  outcome. 
Obviously,  the  presence  of  these  variables  compromises  the  predictive 
value  of  the  statistical  models  presented  in  the  next  section,  and  for  this 
reason  they  were  not  included  once  a  decision  rule  was  formulated.  The 
number  of  months  that  have  passed  since  a  program  satisfied  Milestone 
B  has  been  included  to  determine  whether  overruns  intensify  or  dissipate 
as  more  of  the  program  is  executed. 

Results 

In  explaining  schedule  slips,  the  model  that  explained  the  most 
variance  while  maintaining  parsimony  included  the  sum  of  the  TRL 
shortfalls,  the  number  of  months  since  the  program  passed  Milestone  B, 
and  an  indicator  variable  that  denotes  that  estimation  error  contributed 
to  the  cost  variance  that  has  occurred  to  date.  The  statistical  results  for 
this  model  are  displayed  in  Table  3. 


TABLE  3.  SCHEDULE  SLIPPAGE  REGRESSION  RESULTS 


Variable 

Estimate  (Standard  Error) 

Constant 

-11.50 

(6.41) 

TRL  Sum 

1.54” 

(0.29) 

Months  Since  MS  B 

0.125* 

(0.06) 

Estimating  Error 

10.57* 

(4.06) 

Note.  R2  =  0.55,  N  =  50,  *  P-value  <  0.05,  **  P-value  <  0.001 
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Estimated  Schedule  Slippage  Model 

Slippage  =  a(Sum)  +  b(Months  Since  MS B)  +  ^Estimating )  +  d 

The  model  explains  more  than  half  of  the  variation  in  schedule  slips. 
The  model’s  explanatory  value  strongly  suggests  that  any  decision  rule 
regarding  minimum  technology  readiness  should  incorporate  the  sum 
of  technology  readiness  shortfalls.  The  results  also  reveal  that  schedule 
slippage  worsens  as  the  time  since  Milestone  B  increases,  and  that  cost 
estimating  errors  that  understate  program  expense  contribute  to  sched¬ 
ule  overruns.  This  probably  occurs  because  unpleasant  cost  surprises 
lead  to  the  stretching  of  program  timelines  to  meet  each  calendar  year’s 
budget  targets.  To  evaluate  the  proposed  technology  readiness  rule’s 
value,  the  model’s  efficacy  in  explaining  cost  ratio  was  also  investigated. 
As  the  results  in  Table  4  illustrate,  a  similar  model  also  works  well  in 
explaining  cost  overruns. 


TABLE  4.  COST  OVERRUN  REGRESSION  RESULTS 


Variable 

Estimate  (Standard  Error) 

Constant 

-22.17 

(12.95) 

TRL  Sum 

3.56** 

(0.77) 

Months  Since  MS  B 

0.19 

(0.14) 

TRL  SUM* 

-0.02* 

Months  Since  MS  B 

(0.01) 

Schedule  Error 

19.19* 

(8.55) 

Note.  R2  =  0.53,  N  =  49,  *  P-value  <  0.05,  **  P-value  <  0.001 
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Estimated  Cost  Overrun  Model 

Overrun  =  a(Sum)  +  b(Months  Since  MS B)  + 
c(Sum*  Months  Since  MS B)  +  d(Schedule )  +/ 

In  addition  to  explaining  more  than  half  of  the  variation  in  sched¬ 
ule  slippage,  the  model  also  explains  over  half  of  the  variation  in  cost 
overruns.  Once  again,  the  sum  of  TRL  shortfalls  is  highly  influential  in 
explaining  the  acquisition  outcome.  Cost  overruns  tend  to  increase  in 
severity  as  time  passes,  but  strangely,  the  interaction  between  the  sum 
of  TRL  shortfalls  and  the  number  of  months  since  Milestone  B  has  a 
negative  sign.  This  means  that  the  effects  of  the  sum  of  TRL  shortfalls 
diminish  somewhat  as  time  passes.  However,  evaluating  the  results  from 
both  regressions,  clearly  the  sum  of  TRL  shortfalls  is  the  most  useful  of 
the  variables  considered  in  explaining  cost  and  schedule  overruns. 

To  illustrate  the  results  in  more  concrete  terms,  a  notional  decision 
rule  can  be  specified  and  applied  to  the  systems  in  the  sample.  The  most 
discriminating  decision  rule  in  this  sample  predicts  that  systems  with  a 
sum  of  TRL  shortfalls  above  10  or  a  maximum  technology  shortfall  of  3 
will  cost  significantly  more  than  expected  and  will  experience  a  longer 
cycle  time  than  was  previously  expected.  The  currently  codified  standard 
of  no  maximum  technology  shortfall  of  3  or  higher  was  included  because 
it  is  useful  and  highly  unlikely  to  be  eliminated.  The  average  results  for 
this  rule  are  summarized  in  Table  5.  It  is  especially  noteworthy  that  the 
mean  overrun  for  both  acquisition  outcomes  was  found  to  be  substan¬ 
tially  higher  in  violating  programs  versus  no  violating  programs.  The 
differences  were  found  to  be  statistically  significant  at  the  1  percent  level 
for  schedule  and  at  the  5  percent  level  for  cost. 


TABLE  5.  APPLICATION  OF  THE  PROPOSED  DECISION  RULE 


Quantity  of  Interest 

No  Violation 

Violation 

Mean  Months  Behind  Schedule** 

7.7  mos. 

31.2  mos. 

Mean  Percentage  Cost  Overrun* 

3.2% 

35.5% 

Note.  *  P-value  <  0.05,  **  P-value  <  0.01 
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The  impact  of  the  rule  is  stark— an  average  difference  in  schedule 
overrun  of  over  23.5  months  and  32  percent  higher  relative  costs.  In  fact, 
it  is  reasonable  to  conclude  that  systems  that  do  not  violate  this  decision 
rule  perform,  on  average,  almost  as  expected  with  respect  to  cost,  and 
systems  that  violate  the  rule  generally  have  problems  that  will  lead  to 
multiple  Nunn-McCurdy  and  APB  breaches. 


Possible  Mechanisms  of  Cost  and  Schedule  Growth 

Although  a  causal  link  between  large  technology  gaps  and  cost  and 
schedule  overruns  has  not  been  established  statistically,  identifying 
an  intuitively  appealing  mechanism  that  could  be  causing  these  effects 
would  help  satisfy  us  that  these  are  probably  not  spurious  results.  Dan 
Davis  speculated  that  low  TRLs  might  cause  more  scope  growth  during 
development,  and  showed  that  scope  growth  was  highly  statistically 
significant  in  explaining  development  cost  growth  (Davis,  2010).  Fur¬ 
thermore,  a  quick  analysis  of  a  recently  published  GAO  report  reveals 
that  increases  in  Research  and  Development  (R&D)  costs  are  strongly 
correlated  with  rising  estimated  procurement  costs  (GAO,  2011).  Com¬ 
plexity  could  be  driving  cost  growth  throughout  a  program’s  development 
and  production  phases.  This  mechanism  may  be  equally  valid  for  explain¬ 
ing  systems  that  have  multiple  schedule  overruns.  Using  a  sample  of  70 
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systems  with  current  SARs,  development  cost  increases  led  to  statisti¬ 
cally  significant  increases  in  the  number  of  APB  schedule  breaches,  the 
probability  of  multiple  breaches,  and  the  probability  of  an  APB  breach 
after  Milestone  C.  Therefore,  there  is  statistical  evidence  that  a  large 
technology  gap  could,  through  the  mechanism  of  scope  growth  during  the 
development  phase,  eventually  contribute  to  systemic  cost  and  schedule 
issues  throughout  a  system's  acquisition  cycle.  A  program  that  gets  into 
trouble  early  stays  in  trouble,  and  one  way  to  virtually  ensure  a  troubled 
system  is  to  tackle  too  large  a  technology  gap.  The  figure  shown  here 
summarizes  the  mechanism  graphically. 

FIGURE.  HYPOTHESIZED  MECHANISM  OF  SYSTEMIC  COST  AND 

SCHEDULE  GROWTH 


Now  that  the  predictive  power  of  this  decision  rule  for  recent  sys¬ 
tems  has  been  established,  what  general  guidelines  for  implementing  it 
should  apply?  If  the  rule  is  violated,  the  decision  maker  has  three  options: 

(a)  take  measures  to  bring  the  program  into  compliance  with  the  rule, 

(b)  cancel  or  delay  the  program,  or  (c)  assume  the  risks  associated  with 
large  technology  gaps  without  mitigating  them.  The  implications  of  the 
last  two  options  are  clear,  so  the  emphasis  in  this  article  will  be  on  the 
first  option. 

If  the  system  technology  readiness  gap  exceeds  upper  tolerance 
levels  such  as  the  one  specified  in  the  previous  section,  partially  closing 
the  gap  by  substituting  more  mature  technologies  could  pay  considerable 
dividends.  Of  course,  for  some  systems  such  substitution  would  not  be 
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possible,  but  when  feasible,  it  is  highly  recommended.  Experienced  sys¬ 
tems  engineers  would  work  with  science  and  technology  professionals 
and  cost  estimators  to  make  the  most  cost-effective  trades.  As  the  cost 
and  schedule  performance  of  the  DoD  acquisition  system  improves,  these 
collaborative  efforts  could  potentially  be  instrumental  in  demonstrating 
the  worth  of  systems  engineering.  As  the  DoD  learns  more  about  cost  and 
schedule  performance  at  various  technology  readiness  gaps,  the  technol¬ 
ogy  readiness  decision  rule  can  be  modified  to  reflect  actual  experience. 

Analyses  of  Alternatives  (AoA)  could  play  an  important  role  in 
identifying  potentially  useful  technology  substitutions.  Through  DoD 
Instruction  5000.02,  defense  leaders  have  mandated  the  performance 
of  an  AoA  before  Milestone  A.  An  AoA  is  specifically  intended  to  be 
the  analytical  foundation  for  arriving  at  the  correct  materiel  solution 
when  one  is  required.  In  addition,  AoAs  are  to  offer  an  assessment  of 
the  critical  technology  elements  that  make  up  each  potential  materiel 
solution.  However,  as  the  GAO  has  found,  many  AoAs  do  not  provide  a 
robust  set  of  alternatives  at  the  system  level— much  less  at  the  critical 
technology  element  level.  The  DoD  concurred  with  the  GAO’s  recom¬ 
mendations  for  improvement,  so  it  is  possible  that  AoAs  are  improving 
as  this  is  being  written  (GAO,  2009).  The  DoD  must  prioritize  the  fund¬ 
ing  of  AoAs,  provide  useful  feedback  to  those  who  perform  the  analysis, 
and  emphasize  analysis  at  the  critical  technology  element  level.  Perhaps 
most  importantly,  the  DoD  should  require  that  an  “80  percent  solution” 
and  a  low-budget  option  are  identified  and  analyzed  at  both  the  system 
and  critical  technology  element  level  (Defense  Science  Board,  2009).  The 
DoD  must  deter  the  use  of  an  AoA  merely  to  “support  a  predetermined 
solution”  (GAO,  2007). 

System  Technology  Readiness  Gaps 

Including  an  additional  metric  in  technology  readiness  assessments 
could  help  facilitate  the  efforts  of  systems  engineers  and  program  man¬ 
agers  in  making  technology  trade-offs.  Toward  that  end,  John  Mankins 
has  proposed  adding  Research  and  Development  Degree  of  Difficulty 
(R&D3)  to  technology  readiness  assessments  (Mankins,  2002).  Mankins 
created  five  R&D3  classifications  that  depend  on  how  many  parallel  paths 
of  discovery  researchers  believe  are  necessary  to  ensure  a  reasonable 
probability  of  successful  discovery.  It  is  likely  that  the  number  of  parallel 
paths  required  will  be  positively  correlated  with  total  early  development 
costs.  Therefore,  this  approach  could  also  facilitate  and  standardize 
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the  participation  of  cost  estimators.  By  requiring  researchers  to  submit 
R&D3  classifications  for  each  critical  technology  element  and  then  ask¬ 
ing  cost  estimators  to  provide  a  ballpark  estimate  for  development  costs 
based  on  the  size  of  the  total  technology  gap  and  the  R&D3  classification, 
systems  engineers  and  program  managers  will  be  armed  with  the  infor¬ 
mation  they  need  to  make  reasonably  informed  technology  trade-offs.  Of 
course,  this  metric  could  be  helpful  in  cost  estimation  without  trade-offs 
(option  (c)  at  the  end  of  the  previous  section)  and  could  be  used  to  help 
understand  the  risks  involved  in  developing  systems  with  relatively 
large  technology  gaps.  Although  this  recommendation  almost  certainly 
requires  additional  cost-estimating  staff,  it  is  likely  these  additional  staff 
members  will  pay  for  themselves  by  providing  useful  information  on  how 
to  save  money.  Because  these  estimates  would  be  ballpark  estimates, 
they  would  not  require  the  level  of  analysis  needed  later  in  the  cycle,  and 
the  number  of  new  estimators  would  probably  not  be  prohibitive.  Where 
possible,  we  have  refrained  from  making  recommendations  that  require 
an  up-front  investment  of  increasingly  scarce  resources,  but  here  it  is 
highly  advisable  to  take  the  long  view. 

Although  the  approach  of  evaluating  TRLs  in  isolation  would  be  a 
substantial  improvement  over  existing  practice,  the  long-term  potential 
of  technology  readiness  assessments  that  consider  interfaces  between 
components  has  greater  potential  and  should  be  pursued.  Recently, 
researchers  have  made  progress  in  defining  technology  maturity  metrics 
that  incorporate  the  interface  between  critical  technology  elements 
(Sauser,  Ramirez-Marquez,  Verma,  &  Gove,  2008).  Two  technology 
maturity  metrics  have  been  advanced  in  the  systems  engineering  litera¬ 
ture:  the  interface  readiness  level  (IRL)  and  the  system  readiness  level 
(SRL).  As  their  names  suggest,  IRLs  measure  the  readiness  of  technol¬ 
ogy  that  enables  interoperability  of  two  critical  technology  elements, 
and  the  SRL  is  an  aggregate  number  that  incorporates  both  TRLs  and 
IRLs.  Theoretically,  SRLs  should  be  the  ideal  end  state— allowing  sys¬ 
tems  engineers  to  evaluate  the  contribution  of  each  critical  technology 
element  to  the  functioning  of  the  entire  system  at  the  press  of  a  button. 
However,  SRLs  are  a  concept  in  its  infancy,  and  much  work  needs  to  be 
done  before  transitioning  to  this  technology  readiness  metric. 

Multiple  measurement  issues  arise  when  attempting  to  calculate  an 
SRL  (Kujawski,  2010).  In  the  end,  objections  to  the  calculation  of  SRLs 
amount  to  concerns  over  mixing  arbitrary  subjective  rating  scales  and 
then  aggregating  the  results.  While  these  objections  are  well-founded, 
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there  is  no  reason  why  IRLs  cannot  be  considered  in  assessing  system 
technological  readiness  with  the  eventual  goal  being  more  comprehen¬ 
sive  metrics  for  technology  readiness  assessment. 

IRLs  could  be  used  in  several  ways  without  mixing  the  two  rating 
scales.  First,  minimum  IRL  guidelines  could  be  set  and  considered  when 
making  technology  trades.  In  addition,  overall  IRL  technology  gaps  could 
be  assessed  and  used  to  motivate  technical  substitution  as  we  proposed 
with  TRLs.  Finally,  IRLs  should  play  an  important  role  in  system  cost 
estimation.  With  all  of  these  potentially  worthwhile  applications  associ¬ 
ated  with  IRLs,  the  DoD  should  devote  the  necessary  resources  to  fully 
understanding  the  contribution  of  the  interfaces  to  overall  system  tech¬ 
nology  readiness  and  applying  the  IRL  concept  to  its  complex  systems. 

Although  calculation  of  credible  overall  SRLs  may  be  quite  a  few 
years  off,  useful  information  on  a  system’s  interfaces  could  be  reported 
in  a  modest  amount  of  time.  While  guidelines  that  incorporate  IRLs  are 
being  developed,  the  DoD  can  use  decision  rules  similar  to  those  pro¬ 
posed  in  the  preceding  discussion.  Eventually,  the  DoD  should  be  able  to 
report  a  single  SRL  and  be  able  to  make  trades  based  on  contributions  to 
overall  system  technological  readiness. 


Implications  for  Science  and  Technology 

The  most  obvious  and  potentially  important  implication  of  defer¬ 
ring  the  use  of  relatively  immature  technologies  in  system  development 
relates  to  the  allocation  of  funding  between  Science  and  Technology 
(S&T)  and  programs  of  record.  If  less  R&D  funding  is  to  be  spent  after  a 
system  acquisition  becomes  a  program  of  record,  more  funding  should 
naturally  be  diverted  to  S&T.  This  is  not  a  new  idea— as  far  back  as  2000, 
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the  Defense  Science  Board  recommended  that  S&T  budget  requests 
be  increased  to  almost  3  percent  of  the  total  DoD  budget  submission 
(Morrow,  2000).  In  2003,  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  Pete  Aldridge  set  the  same  target  for  S&T 
funding;  and  in  2007,  the  Pentagon's  Director,  Defense  Research  and 
Engineering  John  Young  argued  that  the  department’s  S&T  funding 
should  be  3  percent  of  the  DoD’s  total  budget  (Davey,  2003;  Chow,  Sil- 
berglitt,  &  Hiromoto,  2009). 

An  increase  in  S&T's  share  of  DoD  funding  flies  in  the  face  of  current 
trends.  In  fiscal  year  2000,  S&T  accounted  for  approximately  3  percent 
of  DoD’s  budget  (Morrow,  2000).  In  the  proposed  fiscal  year  2013  budget, 
S&T  has  slipped  to  2.26  percent  of  DoD’s  allocation  (DoD,  2011).  In  the 
extraordinarily  tight  budget  climate  expected  for  the  foreseeable  future, 
reversing  this  trend  of  decreasing  S&T  funding  as  a  percentage  of  the 
DoD  budget  will  be  very  challenging.  To  gradually  raise  S&T’s  share  of 
total  funding,  the  DoD  could  implement  a  ‘reinvestment’  policy.  That 
is,  as  cost  performance  improves  from  insisting  on  smaller  technology 
maturity  gaps  in  programs  of  record,  the  DoD  could  earmark  a  portion 
of  the  realized  savings  for  use  in  basic  and  applied  research.  Since  total 
defense  S&T  funding  is  less  than  12  percent  of  DoD  procurement  fund¬ 
ing,  demonstrating  the  plausibility  of  such  a  reinvestment  strategy 
is  trivial  (Office  of  the  Under  Secretary,  2012).  If  the  DoD  has  enough 
discipline  to  accomplish  this  ramp-up  in  S&T  funding,  the  Services’ 
technological  edge  will  not  be  diminished  to  an  unacceptable  degree  over 
the  long  haul  despite  obvious  short-term  sacrifices  implied  by  some  of 
these  recommendations. 

If  the  DoD  increases  the  intensity  of  its  S&T  efforts,  the  Services 
must  decide  how  to  prioritize  new  R&D  projects  that  will  become  pos¬ 
sible  with  the  newly  available  resources.  RAND  researchers  working  on 
behalf  of  the  Army  have  made  progress  in  this  area,  and  the  Air  Force  and 
Navy  could  profit  from  adopting  some  of  their  recommendations.  In  the 
most  general  terms,  the  algorithm  they  developed  maximizes  the  number 
of  technology  objectives  prospective  projects  satisfy  while  minimizing 
total  life-cycle  costs  (Chow  et  al.,  2009).  While  the  algorithm  considers 
neither  cycle  time  nor  discovery  risk  at  this  time,  the  general  approach 
holds  promise  and  has  been  under  development  and  revision  since  2002. 
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Conclusions 

A  new  way  of  assessing  a  system’s  TRLs  has  been  demonstrated  to 
be  quite  useful  in  explaining  cost  and  schedule  overruns.  In  particular, 
this  research  shows  that  larger  total  technology  gaps,  as  measured  by  the 
sum  of  the  TRL  shortfalls,  lead  to  drastically  increased  costs  throughout 
the  system’s  useful  life  and  to  more  frequent  and  sizable  schedule  slips. 
Since  the  size  of  the  total  technology  gap  is  a  key  driver,  reducing  the 
technology  gap  by  making  technology  substitutions  where  possible  can 
mitigate  these  risks.  First  and  foremost,  the  DoD  should  enforce  present 
guidelines  and  stop  allowing  critical  technology  elements  with  TRLs 
below  6  to  be  included  in  a  program  after  Milestone  B.  Further,  using  a 
simple  rule  of  thumb  that  aggregates  a  system’s  critical  technology  ele¬ 
ment  readiness  gaps  could  significantly  improve  acquisition  outcomes, 
but  DoD  should  eventually  develop  more  comprehensive  technology 
readiness  metrics  that  have  the  potential  to  reduce  technology  risk  even 
further.  To  implement  these  recommendations,  the  DoD  must  put  the 
right  professionals  in  place— skilled  systems  engineers,  cost  estimators, 
and  researchers  are  vital  to  this  roadmap’s  success.  Finally,  the  DoD 
should  increase  S&T  funding  as  savings  are  realized  from  informed 
technology  substitutions. 
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Review: 

The  Royal  Australian  Navy  (RAN)  operated  submarines  built  and 
supported  by  Great  Britain  for  much  of  the  20th  century.  As  the  RAN’s 
British-built  Oberon  class  submarines  were  reaching  their  mid-life 
point  in  the  early  1980s,  the  RAN  was  finding  it  difficult  and  expensive 
to  support  the  desired  operational  availability  of  their  submarine  fleet. 
In  this  environment,  and  with  much  debate  and  deliberation,  Australia 
decided  that  the  new  submarines  needed  to  replace  the  Oberon  boats 
would  be  built  in  Australia.  The  resulting  Collins  class  submarine 
program  was  the  largest,  most  expensive,  and  most  controversial 
military  project  undertaken  by  Australia’s  defense  community. 

The  authors  of  this  superb  history  of  the  Collins  class  program 
thoroughly  describe  the  numerous  players,  their  intentions,  and  their 
interactions  during  the  20  years  from  the  beginnings  of  the  program  to 
the  delivery  of  the  sixth  and  final  submarine  in  the  class.  The  program 
produced  not  only  one  of  the  largest  and  most  capable  diesel  submarines 
in  the  world,  it  also  created  a  new  national  industry.  However,  it  was 
marked  with  technical  difficulties  and  political  intervention.  As  the 
authors  state,  “It  is  a  story  of  heroes  and  villains,  grand  passions, 
intrigue,  lies,  spies  and  backstabbing”  (p.  xviii). 

The  authors  tell  their  story  in  four  parts: 

•  The  early  years  of  debate  on  whether  Australia  could  actually 
build  submarines,  followed  by  the  solicitation  and  awarding 
of  a  design  and  construction  contract  to  an  alliance  of  several 
companies.  Acquisition  professionals  will  find  informative  the 
process  used  to  set  requirements,  the  contracting  structure,  and 
the  interactions  between  the  buyer  and  the  seller. 

•  The  first  few  years  of  design  and  construction,  when  enthusiasm 
and  newness  created  an  atmosphere  of  cooperation  and 
progress.  However,  there  were  several  issues  arising  between 
the  corporations  involved  in  the  prime  contractor  partnership. 
The  design  developed  by  the  Swedish  shipbuilder,  Kockums,  was 
based  on  a  Swedish  Navy  boat  whose  operational  capabilities 
were  very  different  from  those  desired  by  Australia.  Technical 
problems  began  to  emerge,  especially  in  the  combat  system,  and 
construction  lagged.  Acquisition  professionals  will  benefit  from 
the  description  of  the  combat  system  contract  environment  as 
well  as  the  creation  of  a  new  cooperative  venture  and  a  green¬ 
field  shipyard. 
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•  A  several-year  period  when  the  bloom  fell  off  the  rose.  This  period 
was  marked  by  increasing  technical  problems  and  construction 
delays.  The  public’s  perception  of  the  program  problems  was 
inflamed  by  several  disparaging  media  articles.  The  original 
partnership  changed  dramatically  as  Australia  assumed  the 
ownership  of  the  Australian  Submarine  Corporation  (ASC). 
Acquisition  professionals  will  find  enlightening  the  process 
through  which  the  contracting  arrangements  changed 
and  the  recognition  that  the  government  had  substantially 
underestimated  the  risks  in  undertaking  such  a  complex  program 
for  the  first  time. 

•  The  last  several  years  of  the  program  when  the  Collins 
submarines  finally  became  operationally  capable  with  the  help 
of  the  United  States  Navy.  Problems  during  this  time  period 
switched  from  designing  and  building  the  submarines  to 
providing  the  logistics  support  needed  to  attain  operational 
goals.  This  problem  persists  today  as  marked  by  new  studies 
seeking  to  identify  the  inherent  problems  in  supporting  the 
Collins  class  submarines  and  the  future  costs  needed  to  keep 
the  boats  operationally  ready  during  the  second  halves  of  their 
operational  lives. 
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approach  in  either  content  or  language. 

Format 
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abstract  (120  words  or  less),  two-line  summary,  list  of  key  words  (five  words  or  less),  body 
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e-mail  addresses,  as  well  as  telephone  and  fax  numbers.  The  letter  should  verify  that  the 
submission  is  an  original  product  of  the  author;  that  it  has  not  been  previously  published  in 
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computer  application  programs,  and  file  names  used  on  enclosed  CDs,  e-mail  attachments, 
or  other  electronic  media. 


COPYRIGHT 


The  Defense  ARJ  is  a  publication  of  the  United  States  Government  and  as  such  is 
not  copyrighted.  Because  the  Defense  ARJ  is  posted  as  a  complete  document  on  the  DAU 
homepage,  we  will  not  accept  copyrighted  manuscripts  that  require  special  posting 
requirements  or  restrictions.  If  we  do  publish  your  copyrighted  article,  we  will  print  only 
the  usual  caveats.  The  work  of  federal  employees  undertaken  as  part  of  their  official  duties 
is  not  subject  to  copyright  except  in  rare  cases. 

Web-only  publications  will  be  held  to  the  same  high  standards  and  scrutiny  as  articles 
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Washington,  DC).  Contributors  will  be  required  to  submit  a  copy  of  the  writer’s  permission 
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